#2018-03-12 15:17:54 ghadi our depstar tool just got an awesome logo courtesy @quephird #2018-03-12 15:18:53 alexmiller I’ll pretend I didn’t see that :) #2018-03-12 15:21:01 alexmiller does quephird work with you? #2018-03-12 15:21:16 ghadi yes @alexmiller #2018-03-12 15:21:22 alexmiller aw, cool. she’s great. #2018-03-12 15:22:00 alexmiller ;) #2018-03-12 15:22:30 alexmiller I meant to put all that in pm to Ghadi :) #2018-03-12 15:41:41 devn 🙂 #2018-03-12 17:39:38 alexmiller linked from the readme too if you’re trying to find it later #2018-03-12 17:54:53 seancorfield I think that last line should be cljs-test-runner, not clj-test-runner (link is correct, text is not). #2018-03-12 17:55:27 alexmiller thx #2018-03-12 18:33:45 ghadi depstar is one word @alexmiller - thx for the mention #2018-03-12 18:34:09 alexmiller it’s a wiki, fix as you like :) #2018-03-12 18:34:37 alexmiller and if it’s misconfigured for edits, let me know #2018-03-12 18:57:53 ghadi I'm getting a 403 when I clone then push @alexmiller #2018-03-12 18:58:49 ghadi (I can't remember if there is supposed to be an edit button in the UI -- but there isn't one visible) #2018-03-12 19:00:10 alexmiller I’m sure it’s set wrong - seems like the default is always wrong :) #2018-03-12 19:00:19 alexmiller @ghadi clone then push what? #2018-03-12 19:01:16 ghadi the wiki repo #2018-03-12 19:01:55 ghadi I think I was supposed to edit in the UI #2018-03-12 19:05:43 alexmiller well that’s how normal people do it :) I’m getting Stu to change the setting #2018-03-12 21:27:49 alexmiller other options like? #2018-03-12 21:28:08 alexmiller oh, I see what you mean #2018-03-12 21:28:16 alexmiller good idea, feel free to jira / patch #2018-03-12 21:28:58 alexmiller that affects memoization of course so would need to probably expose the scrape function with a better name #2018-03-12 21:34:47 dominicm tbh, if I was being very specific, it might be just that I need the version that Stu originally created where the different config files get labelled. #2018-03-12 21:36:56 alexmiller well that idea is still in play for several reasons #2018-03-12 21:37:10 alexmiller prob the same reasons you’re interested in it #2018-03-12 21:37:54 alexmiller that’s going to be a mildly painful change though across the script and tda #2018-03-12 21:44:19 dominicm alpha, baby 🙂 #2018-03-12 21:51:10 dominicm (I can picture Austin powers saying that) #2018-03-12 22:07:24 mhuebert can a github dep in deps.edn specify the root path (into the repo) for the desired library? #2018-03-12 22:08:07 mhuebert I have been publishing more than one library from a single repo #2018-03-12 22:08:14 alexmiller Yeah with :deps/root #2018-03-12 22:08:35 mhuebert would that be a relative or absolute path? #2018-03-12 22:08:43 mhuebert /my-lib #2018-03-12 22:09:47 alexmiller Relative in the repo #2018-03-12 22:09:57 alexmiller To the base dir #2018-03-12 22:12:41 dominicm @mhuebert fyi, when a transitive dependency does :local/root, it is relative to where you're running clj from. So you can't put things in sub-dirs, it has to be flat. #2018-03-12 22:12:58 dominicm We're doing this, you're not alone it's good fun 🙂 #2018-03-12 22:14:15 mhuebert hmm. ok. i am not using :local/root anywhere at the moment #2018-03-12 22:14:25 mhuebert but I am seeing:
Error building classpath. nil
java.lang.NullPointerException
at clojure.string$starts_with_QMARK_.invokeStatic(string.clj:364)
at clojure.string$starts_with_QMARK_.invoke(string.clj:360)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:48)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:43)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_fetch.invokeStatic(impl.clj:64)
at clojure.tools.gitlibs.impl$git_fetch.invoke(impl.clj:61)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:105)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
#2018-03-12 22:14:36 mhuebert for the dep
lark/tools {:git/url ""
:sha "cc1305534b1371df3962fd46503dd0478d81a08a"
:deps/root "tools"}
#2018-03-12 22:15:00 mhuebert I can run deps from that directory locally (“tools”), it has a deps.edn file #2018-03-12 22:15:13 dominicm fwiw, that should work, I used it at some point, 100% #2018-03-12 22:15:24 mhuebert hmm. #2018-03-12 22:18:21 mhuebert #2018-03-12 22:18:25 seancorfield @mhuebert FWIW, I just ran (! 581)-> clj -Sdeps '{:deps {lark/tools {:git/url ""
:sha "cc1305534b1371df3962fd46503dd0478d81a08a"
:deps/root "tools"}}}
'
and it worked fine and gave me a REPL. #2018-03-12 22:18:48 mhuebert hmm #2018-03-12 22:19:03 mhuebert ok thanks for trying that. maybe i need to update my clj #2018-03-12 22:21:52 mhuebert well that’s weird. that exact command doesn’t work on my machine, with latest clj from brew #2018-03-12 22:24:28 seancorfield I'm on the latest clojure 1.9.0.358 already installed #2018-03-12 22:24:42 seancorfield (output from brew upgrade clojure) #2018-03-12 22:24:47 mhuebert yeah, me too #2018-03-12 22:25:17 seancorfield I created a new folder mhuebert and dropped into there to run that command with no deps.edn file present, just to confirm it works on its own. #2018-03-12 22:25:47 seancorfield If you have deps.edn, it will read that and maybe the NPE is coming from something else there? #2018-03-12 22:26:05 mhuebert yeah I tried the same thing, ran it from a fresh directory in a new terminal window #2018-03-12 22:26:18 mhuebert mysteries of the universe #2018-03-12 22:26:27 seancorfield Maybe something in your ~/.clojure/deps.edn then? #2018-03-12 22:27:02 mhuebert appears to be all commented out #2018-03-12 22:29:17 mfikes Try -Srepro to be sure #2018-03-12 22:30:06 mhuebert that produces a repl #2018-03-12 22:30:22 mfikes You must have something in your deps.edn chain 🙂 #2018-03-12 22:30:59 mhuebert somehow the url passed to call-with-auth is nil, when it’s trying to fetch from github #2018-03-12 22:31:03 mfikes Other hunch: Maybe a cache gone bad? and it needs to be forced #2018-03-12 22:31:39 mhuebert (defn- call-with-auth
([^GitCommand command]
(call-with-auth
(.. command getRepository getConfig (getString "remote" "origin" "url"))
command))
([^String url ^GitCommand command]
(if (and (instance? TransportCommand command)
(not (str/starts-with? url "http")))
(.. ^TransportCommand command (setTransportConfigCallback @ssh-callback) call)
(.call command))))
#2018-03-12 22:31:45 mhuebert third from bottom, the str/starts-with? #2018-03-12 22:31:50 mhuebert is there some caching somewhere that I can clear? #2018-03-12 22:32:12 mfikes I''d try without -Srepro and add -Sforce (but I'm speculating) #2018-03-12 22:34:10 mhuebert oh, I maybe misread/communicated, I tried -Srepro by itself earlier #2018-03-12 22:34:15 mhuebert clj -Srepro -Sforce -Sdeps '{:deps {lark/tools {:git/url ""
:sha "cc1305534b1371df3962fd46503dd0478d81a08a"
:deps/root "tools"}}}'
#2018-03-12 22:34:34 mhuebert still null pointer. hmm. #2018-03-12 22:39:19 mhuebert i wonder if it could have anything to do with git auth #2018-03-12 22:41:10 mhuebert well this will have to wait for another day #2018-03-13 07:31:40 mhuebert trying clojure.tools.gitlibs, I get a failure loading slf4j:
user=> (require '[clojure.tools.gitlibs :as gl])
nil
user=> (gl/resolve "" "cc1305534b1371df3962fd46503dd0478d81a08a")
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
NullPointerException clojure.string/starts-with? (string.clj:364)
#2018-03-13 07:33:20 mhuebert adding slf4j-simple to class path resolves that warning but doesn’t affect the error #2018-03-13 07:41:39 mhuebert maybe there was some corrupt thing there #2018-03-13 07:42:35 mhuebert unfortunately I can’t find a way to reproduce this now #2018-03-13 07:43:27 mhuebert oh, now i’ve got it. found the old dir in the trash. #2018-03-13 07:44:11 mhuebert well, that is simple. #2018-03-13 07:46:34 mhuebert Reproduction steps:
1. create an empty directory at ~/.gitlibs/_repos/github.com/<user>/<repo>
2. try loading any version of #2018-03-13 07:47:22 mhuebert my original empty dir was somehow created by gitlibs, maybe while I was trying to figure out how to properly specify a root dir #2018-03-13 13:23:15 dominicm I need to write my jira from last yesterday #2018-03-13 13:23:18 dominicm maybe #2018-03-13 20:44:22 bhauman if clojure.main referenced a *main-repl-fn* #2018-03-13 20:45:43 bhauman then we could set that with an -e "(set! clojure.main/*main-repl-fn* rebel-readline.main/repl)" #2018-03-13 20:46:59 bhauman and then bobs your uncle #2018-03-13 20:48:56 alexmiller clojure.main is in Clojure, so this would require a change in Clojure 1.10. I think ultimately we’ll want a change that does something like that. But I’d like to look at a shorter range version. #2018-03-13 20:50:14 dominicm I feel like I'm missing some context? #2018-03-13 20:54:12 bhauman @dominicm I was just trying to think of a lightweight way for clj to inject rebel-readline, adding an inline dep and extra e arg would be easy #2018-03-13 20:55:29 seancorfield Currently, I have :rebel {:extra-deps {com.bhauman/rebel-readline {:mvn/version "RELEASE"}}
:main-opts ["-m" "rebel-readline.main"]}
in my ~/.clojure/deps.edn file and just say clj -A:rebel when I want a rebel REPL... #2018-03-13 20:55:46 dominicm yeah, I have that, but I namespaced it as :user/rebel. #2018-03-13 20:55:49 bhauman but that doesn't help clj #2018-03-13 20:56:00 dominicm no? #2018-03-13 20:56:14 dominicm OH. You mean for including rebel in clj? #2018-03-13 20:56:19 bhauman yes 🙂 #2018-03-13 20:56:33 dominicm 🤔 does rebel have any dependencies #2018-03-13 20:57:24 bhauman extremely flat, tools reader is the only one I still want to get rid of #2018-03-13 20:57:53 bhauman that requires me writing my own indenting code #2018-03-13 20:58:11 bhauman after that there would be no deps besides Jline of course #2018-03-13 20:58:35 dominicm I mean, you could use Mr. Anderson to package it? #2018-03-13 21:00:08 bhauman oh interesting, didn't know about that #2018-03-13 21:08:45 dominicm Shading is a pretty cool thing 🙂 #2018-03-13 22:15:51 dominicm @seancorfield do you have your handy snippet for clj + socket server? #2018-03-13 22:16:35 seancorfield This one? :socket {:jvm-opts ["-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl}"]}
#2018-03-13 22:19:29 dominicm Yep #2018-03-13 22:20:03 dominicm clj -J-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl} -A:dev:build is giving me
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol
at clojure.lang.RT.seqFrom(RT.java:547)
at clojure.lang.RT.seq(RT.java:527)
at clojure.lang.APersistentMap.cons(APersistentMap.java:40)
at clojure.lang.RT.conj(RT.java:667)
at clojure.core$conj__4932.invokeStatic(core.clj:85)
at clojure.core$merge$fn__5498.invoke(core.clj:3029)
at clojure.core$reduce1.invokeStatic(core.clj:936)
at clojure.core$reduce1.invokeStatic(core.clj:926)
at clojure.core$merge.invokeStatic(core.clj:3028)
at clojure.core.server$parse_props$fn__8321.invoke(server.clj:149)
at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
at clojure.core.protocols$fn__7665.invokeStatic(protocols.clj:168)
at clojure.core.protocols$fn__7665.invoke(protocols.clj:124)
at clojure.core.protocols$fn__7620$G__7615__7629.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__7642.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__7642.invoke(protocols.clj:75)
at clojure.core.protocols$fn__7594$G__7589__7607.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6704)
at clojure.core.server$parse_props.invokeStatic(server.clj:143)
at clojure.core.server$start_servers.invokeStatic(server.clj:154)
at clojure.core.server$start_servers.invoke(server.clj:154)
at clojure.lang.Var.invoke(Var.java:381)
at clojure.lang.RT.doInit(RT.java:486)
at clojure.lang.RT.<clinit>(RT.java:333)
... 1 more
java.lang.NoClassDefFoundError: Could not initialize class clojure.lang.RT
at clojure.lang.PersistentQueue.pop(PersistentQueue.java:113)
at clojure.lang.PersistentQueue.pop(PersistentQueue.java:25)
at clojure.lang.Agent$Action.doRun(Agent.java:149)
at clojure.lang.Agent$Action.run(Agent.java:163)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
which is interesting #2018-03-13 22:21:43 dominicm Works in ~/.clojure/deps.edn though. #2018-03-13 22:21:52 seancorfield You need "..." around it. #2018-03-13 22:22:02 seancorfield clj -J"-Dclojure..." #2018-03-13 22:22:41 seancorfield Or this clj -J-Dclojure.server.repl="{:port,5555,:accept,clojure.core.server/repl}"
#2018-03-13 22:24:03 dominicm I did the latter originally. But I got the above error. #2018-03-13 22:32:44 seancorfield (! 614)-> clj -J-Dclojure.server.repl="{:port,5555,:accept,clojure.core.server/repl}"
Clojure 1.9.0
user=> ^D
Tue Mar 13 15:32:30
(sean)-(jobs:0)-(~/clojure/lsnape)
(! 615)-> clj -J-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl}
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol
#2018-03-13 22:33:03 seancorfield I get the exception without the quotes -- but it works with quotes as seen above. #2018-03-13 22:39:12 dominicm I'll explore it later. ¯\(ツ)/¯ #2018-03-14 03:51:44 jebberjeb I’ve just been doing $> JAVA_OPTS="-Dclojure..." clj. Same for lein #2018-03-14 03:53:34 jebberjeb My thinking was that it’s nice because it doesn’t depend on how a particular thing handles command line args? #2018-03-14 07:50:43 dominicm I didn't know that worked for clj, neat. #2018-03-14 11:52:18 niclasnilsson I want to get these types of things working, and I think I’ve tried quoting and escaping the things after clj in all ways possible, but I bet there is a way I haven’t tried, since I can’t get it to work:
#!/usr/bin/env clj -Sdeps '{:deps {clj-time {:mvn/version "0.14.2"}}}'
(require '[clj-time.core :as t])
(println (str "Time is now " (t/now)))
I either get a "Error building classpath. Don't know how to create ISeq from: clojure.lang.Symbol" or a "Error while parsing option "--config-data {:deps": java.lang.RuntimeException: EOF while reading" for a combinations that I’ve tried. I bet someone knows how to do this properly? #2018-03-14 12:15:58 dominicm @niclasnilsson what OS are you on? That won't work for Linux. #2018-03-14 12:47:32 niclasnilsson @dominicm MacOS #2018-03-14 12:48:32 niclasnilsson Do you mean like “there is no way to make it work”? #2018-03-14 12:51:34 dominicm There's a limitation on how shebangs are read. It doesn't apply to mac though.
You may find it works if you don't use env #2018-03-14 13:12:48 niclasnilsson Ah, ok. Will try again then. Thanks! #2018-03-14 16:19:45 seancorfield Also: try using , instead of space in the deps string. #2018-03-14 16:30:37 dominicm @seancorfield Yes, I think you might have the right of it there. ' doesn't have have the same effect in shebangs as on the shell. #2018-03-14 17:09:43 seancorfield Confirmed that the following works @niclasnilsson (! 622)-> ./time.sh
Time is now 2018-03-14T17:09:15.652Z
Wed Mar 14 10:09:15
(sean)-(jobs:0)-(~/clojure/bin)
(! 623)-> cat time.sh
#!/usr/bin/env clj -Sdeps {:deps,{clj-time,{:mvn/version,"0.14.2"}}}
(require '[clj-time.core :as t])
(println (str "Time is now " (t/now)))
On macOS #2018-03-14 19:56:50 dominicm I didn't know BSD did it right too. #2018-03-14 17:11:11 niclasnilsson Ah… I wasn’t even close to try that combination. Neat, thanks, @seancorfield! #2018-03-14 17:12:27 niclasnilsson Will try myself later tonight, when I get back to this. #2018-03-15 06:20:14 niclasnilsson Worked fine! Thanks, @seancorfield #2018-03-15 12:16:50 delaguardo Is it possible to redefine local maven repo location with tools-deps? #2018-03-15 12:21:24 alexmiller Yes, I think the key is :mvn/local-repo #2018-03-15 12:21:37 alexmiller At the root of deps.edn #2018-03-15 12:23:20 alexmiller Yeah, that’s it #2018-03-15 12:24:43 delaguardo Cool! thanks. will check this out #2018-03-15 12:37:19 mpenet any eta for TDEPS-9, or if another approach is considered for that issue? #2018-03-15 12:44:22 mpenet having a blank: what's the way to get a dependency from a subdir in a git repo with multiple dependencies ? #2018-03-15 12:44:49 mpenet I think I got that answered a while back, but I can't remember the answer (and well... slack history is not helping) #2018-03-15 12:51:06 niclasnilsson Sometimes I wonder how Slack managed to take over the world so much. The history problem sucks for large open things like Clojurians. #2018-03-15 14:24:23 seancorfield Off-road topic for this channel -- take it to #community-development if you want to discuss Slack/history/etc. #2018-03-15 12:47:27 mpenet it might be :deps/root reading the source #2018-03-15 12:51:17 alexmiller yes #2018-03-15 12:55:48 alexmiller on TDEPS-9, no eta. I’m consulting right now so only getting a few hrs a week on tdeps. The windows installer is my top priority at the moment, hoping to get that done this week. #2018-03-15 14:25:38 delaguardo I found strange bug -
15:24 $ clj -Srepro -Sdeps "{:mvn/local-repo \".m2/repository\"}" -Spom
Downloading: org/clojure/clojure/1.9.0/clojure-1.9.0.pom from
Downloading: org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.pom from
Downloading: org/clojure/pom.contrib/0.2.2/pom.contrib-0.2.2.pom from
Downloading: org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.pom from
Downloading: org/clojure/clojure/1.9.0/clojure-1.9.0.jar from
Downloading: org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar from
Downloading: org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar from
Unknown option: "--config-data"
#2018-03-15 14:26:43 delaguardo It happens only when I’m trying to regenerate pom.xml file with local-repo #2018-03-15 14:28:28 alexmiller the local-repo is an absolute path btw, although I don’t think that’s the issue you’re seeing #2018-03-15 14:29:31 delaguardo but pom.xml is not generated in that case #2018-03-15 14:29:45 delaguardo Will do #2018-03-15 14:32:18 alexmiller there’s some duplicated code in the make classpath and pom gen stuff and I don’t think the latter was updated the last time the former changed #2018-03-15 15:23:44 seancorfield FWIW, that command ultimately becomes this + /usr/bin/java -Xmx256m -classpath /usr/local/Cellar/clojure/1.9.0.358/libexec/clojure-tools-1.9.0.358.jar clojure.main -m clojure.tools.deps.alpha.script.generate-manifest --config-files /usr/local/Cellar/clojure/1.9.0.358/deps.edn,deps.edn --gen=pom --config-data '{:mvn/local-repo ".m2/repository"}'
and that's what complains about --config-data #2018-03-15 15:25:17 alexmiller yeah, I’m aware - the generate-manifest hasn’t tracked changes in make-classpath #2018-03-15 17:42:43 mhuebert has anyone seen something like this when running -Spom?
clj -Spom
Error generating pom manifest: called children on a leaf node
java.lang.Exception: called children on a leaf node
at clojure.zip$children.invokeStatic(zip.clj:80)
at clojure.zip$append_child.invokeStatic(zip.clj:223)
at clojure.zip$append_child.invoke(zip.clj:223)
at clojure.tools.deps.alpha.gen.pom$xml_update$fn__2247.invoke(pom.clj:94)
at clojure.tools.deps.alpha.gen.pom$xml_update.invokeStatic(pom.clj:87)
at clojure.tools.deps.alpha.gen.pom$xml_update.invoke(pom.clj:83)
at clojure.tools.deps.alpha.gen.pom$replace_deps.invokeStatic(pom.clj:98)
at clojure.tools.deps.alpha.gen.pom$replace_deps.invoke(pom.clj:96)
at clojure.tools.deps.alpha.gen.pom$sync_pom$fn__2265.invoke(pom.clj:120)
at clojure.tools.deps.alpha.gen.pom$sync_pom.invokeStatic(pom.clj:117)
at clojure.tools.deps.alpha.gen.pom$sync_pom.invoke(pom.clj:112)
at clojure.tools.deps.alpha.script.generate_manifest$_main.invokeStatic(generate_manifest.clj:49)
at clojure.tools.deps.alpha.script.generate_manifest$_main.doInvoke(generate_manifest.clj:27)
#2018-03-15 17:47:25 alexmiller what’s your existing pom.xml? #2018-03-15 17:48:11 alexmiller does it have a <dependencies>? #2018-03-15 17:54:44 mhuebert let me see #2018-03-15 17:55:09 mhuebert yes it does #2018-03-15 17:56:37 mhuebert I can run -Spom if I remove the existing pom.xml #2018-03-15 18:20:50 alexmiller well if you can capture the case when it fails, would be happy to take a ticket for that #2018-03-15 18:21:00 alexmiller the pom.xml when it fails that is #2018-03-15 20:20:28 mhuebert here is a minimal pom.xml which produces the error:
<?xml version="1.0" encoding="UTF-8"?>
<!-- generated by shadow-cljs pom, do not edit -->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>editor</groupId>
<artifactId>editor</artifactId>
<version>0.0.1</version>
<name>editor</name>
<dependencies>
<dependency>
<groupId>thheller</groupId>
<artifactId>shadow-cljs</artifactId>
<version>2.2.9</version>
</dependency>
</dependencies>
</project>
#2018-03-15 20:20:49 mhuebert I can create a ticket for that if it would be helpful #2018-03-15 20:21:32 alexmiller yes, also would help to know which jdk you’re using #2018-03-15 20:22:01 alexmiller java -version #2018-03-15 20:23:54 mhuebert java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
#2018-03-15 20:27:56 alexmiller I also assume you’re using latest clj (`clj -Sverbose`) ? #2018-03-15 20:28:18 alexmiller latest is 1.9.0.358 #2018-03-15 20:54:55 mhuebert yes #2018-03-15 20:55:16 alexmiller k, if I get a chance I’ll look at it tomorrow #2018-03-15 20:55:57 alexmiller this doesn’t seem much different than everything else I’ve done that works fine so not sure what the deal is #2018-03-15 21:07:40 seancorfield Looks like it's the comment that causes problems @mhuebert /cc @alexmiller #2018-03-15 21:07:46 alexmiller I was just about to say that :) #2018-03-15 21:08:27 alexmiller there’s an assumption in the code about the doc structure that’s being foiled there #2018-03-15 21:08:54 seancorfield If you move the comment inside <project> it's fine. Or delete it. #2018-03-15 21:09:21 seancorfield (sorry, it made me too curious to resist trying to figure out what was causing it) #2018-03-15 21:09:46 alexmiller I will fix it, but that’s a workaround in the meantime #2018-03-16 13:06:02 dnolen @alexmiller the behavior of -R seems a big strange to me, shouldn’t it compose any aliases + your local deps.edn deps? #2018-03-16 13:37:18 alexmiller -R is solely about selecting aliases to be used when resolving deps. The aliases it can select from are those produced by the combined deps.edn files in play, which typically includes your install, user, local deps.edn (and -Sdeps passed on command line) #2018-03-16 13:37:31 alexmiller aliases are not pulled from transitive deps #2018-03-16 13:39:22 alexmiller I’m not sure that would even be possible based on the order in which things are resolved #2018-03-17 08:06:35 dominicm I've been using the :user/ namespace for aliases on my home deps file, with the idea that it will prevent collisions. #2018-03-18 14:40:01 hypirion So I have a (pruned) dependency tree that looks like this
.
├── a 1
│ └── b 1
│ └── x 2
│ └── y 1
└─── c 1
└── x 1
└── z 1
and my result when doing clj -Stree is the following:
org.clojure/clojure 1.9.0
org.clojure/core.specs.alpha 0.1.24
org.clojure/spec.alpha 0.1.143
c path-to-c 1
a path-to-a 1
b path-to-b 1
x path-to-x 2
y path-to-y 1
z path-to-z 1
I'm guessing this is a bug? I would love to send a reproducible example, but it's a bit hard without polluting my GitHub profile. Maybe adding an option to include local git repositories for :git/url would be an idea? #2018-03-18 15:10:12 gfredericks would a gist pollute your github profile? #2018-03-18 15:13:06 alexmiller some things that would be helpful:
- version info from clj -Sverbose (I’m assuming you’re on latest, but if not, then there were significant changes to resolution in 1.9.0.309)
- output from adding -Sdeps '{:aliases {:v {:verbose true}}}' -R:v #2018-03-18 15:14:06 alexmiller with that I probably don’t need the reproducible deps.edn #2018-03-18 15:16:59 alexmiller this is the trickiest kind of version resolution issue - when you have different versions of a dep and the lib sets (not just the versions) are changing across them. #2018-03-18 15:17:52 alexmiller I’m assuming that what you expect here is to have a1, b1, x2, y1, c1 as the output (and no z1) #2018-03-18 15:31:09 hypirion version = 1.9.0.358
install_dir = /usr/local/lib/clojure
config_dir = /home/jeannikl/.clojure
config_paths = /usr/local/lib/clojure/deps.edn /home/jeannikl/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/674722798.cp
Nothing in /usr/local/lib/clojure/deps.edn or /home/jeannikl/.clojure/deps.edn
@gfredericks: That's a good idea, forgot about gists #2018-03-18 15:32:27 hypirion Also yes, I'd expect z1 to not be present #2018-03-18 15:46:24 alexmiller which leads me to believe its most likely a path comparison issue with git deps #2018-03-18 15:46:49 alexmiller the verbose output should be sufficient for me to track that down #2018-03-18 15:49:41 hypirion Huh, it seems like it depends on the order of the elements in the map. #2018-03-18 15:49:54 hypirion Almost done with the gists now, one sec #2018-03-18 15:59:53 hypirion I'm still a bit unsure about how the algorithm works, but I think it should be possible to walk the dependency tree at the end. If you can't reach a dependency by walking the tree, then don't include it as a dependency you need. #2018-03-18 16:03:59 alexmiller I have to run but will look at it later #2018-03-18 16:04:48 alexmiller That’s being done but it relies on comparing deps up the path #2018-03-18 16:06:08 alexmiller If the same dep has two different representations then that path matching will fail. That shouldn’t happen, but that’s my suspicion #2018-03-18 16:16:26 alexmiller Interesting, well I guess I was just lucky :) #2018-03-18 16:22:23 hypirion heh, yeah. Dependency algorithms are hard to get right #2018-03-20 16:04:30 ghadi Does anyone use cljfmt via clj? #2018-03-20 17:00:30 delaguardo I can use it like that
clj -A:cljfmt -e "(require '[leiningen.cljfmt])" -e "(leiningen.cljfmt/cljfmt {:source-paths [\"src\"] :test-paths [\"test\"]} \"check\")"
with an alias inside on deps.edn
#2018-03-20 17:00:36 delaguardo :cljfmt {:extra-deps {leiningen {:mvn/version "2.8.1"}
lein-cljfmt {:mvn/version "0.5.7"}}}
#2018-03-20 17:21:16 ghadi Hero, thanks #2018-03-21 20:45:51 kidpollo To be able to use :git/url sources the target repo has to also have a deps.edn file right? #2018-03-21 20:46:48 alexmiller it needs to either not have deps or declare them with an understood manifest format #2018-03-21 20:47:05 alexmiller currently tools.deps understands deps.edn and pom.xml #2018-03-21 20:47:55 alexmiller or you can force deps.edn semantics with no deps by using :deps/manifest :deps #2018-03-21 20:47:56 kidpollo I see. What about local mvn sources. I am having a hard time getting Artifactory sources working with deps :S #2018-03-21 20:48:11 alexmiller what does “local mvn sources” mean? #2018-03-21 20:48:53 kidpollo I am looking for a workaround for the “no username password sources supported” #2018-03-21 20:50:21 kidpollo thanks for pointing that out @danielcompton #2018-03-21 20:50:22 alexmiller you could mirror to an s3 repo accessed via aws creds #2018-03-21 20:50:30 alexmiller that’s supported #2018-03-21 20:50:57 kidpollo ok thanks! #2018-03-21 21:20:39 danielcompton I'm eagerly awaiting TDEPS-9 to be fixed, a few https://deps.co customers need it before they can start using it #2018-03-21 21:21:10 danielcompton @alexmiller is there anything I can do to help there, or is it just waiting for review? #2018-03-21 21:21:35 alexmiller it’s just down my queue a bit. We need it for Datomic too. #2018-03-21 21:22:18 alexmiller I’m on a consulting project these days so only getting a few hours of Clojure time per week atm. My priority right now is the Windows installer which is getting close. #2018-03-21 21:22:33 alexmiller Any testing or feedback on TDEPS-9 would be helpful. #2018-03-21 21:28:17 danielcompton k, will take a look and see if I can verify that it works #2018-03-21 21:48:39 dominicm @alexmiller which patch is it you want testing/feedback on? Or both? #2018-03-21 21:52:07 alexmiller Don’t know, I haven’t looked at it at all #2018-03-21 21:52:35 alexmiller my only constraint is - no passwords in deps.edn files :) #2018-03-21 21:54:12 dominicm that seems like it discounts patch a. #2018-03-21 21:54:47 alexmiller well there you go #2018-03-21 22:06:39 dominicm @danielcompton I must say, I don't know if my patch works under java 9. I know reflection changed somehow in java 9, but I haven't taken the time to understand that change. #2018-03-21 22:07:42 danielcompton I'll give that a go too 🙈 #2018-03-21 22:07:56 dominicm You've made yourself the expert on Java 9 by blogging about it one time 😛 #2018-03-21 22:13:37 alexmiller Based on what I know, it probably does or can be made to in the context of clj #2018-03-21 22:15:28 alexmiller The best way to test stuff like this is to hack your active clojure script to prepend your tools.deps.alpha/src/main/clojure dir to your lib path #2018-03-21 22:16:06 alexmiller Then you pick up all local changes immediately and can interactively test clj #2018-03-22 16:03:44 ghadi git-related
- https authentication is not supported yet by clj -- I wrote a patch that is waiting for eyes upstream
- ssh authentication via Java is a usability trashfire. Works easily for some, doesn't work for others. I started rewriting tools.gitlibs (the underlying library) to shell out to git, but I'm not sure it is viable because the solution must work across Windows to be accepted upstream.
- must ensure ssh-keyscan -H <host> >> ~/.ssh/known_hosts because you cannot accept unseen ssh hostkeys (clj is non-interactive)
- esoteric .ssh/config options can break cloning because the underlying java ssh libraries reads the config file poorly
- ed25519 keys supported via ssh-agent, but not via ~/.ssh/config entries
i wrote that internally, thought it would be a useful summary of some usability issues here #2018-03-22 16:07:51 alexmiller If you want to make a troubleshooting page on wiki or site, go for it #2018-03-22 16:16:03 ghadi wiki page is a great idea #2018-03-22 16:19:45 dominicm Report from a training session we did with clj using the linux installer. The jars were created as root read-only! That makes the scripts broken for most users to our knowledge. #2018-03-22 18:38:54 ghadi what did you expect to happen? #2018-03-22 19:14:56 dominicm The jars would be readable by everyone. #2018-03-22 19:15:34 ghadi oh they're only readable by root? #2018-03-22 19:15:59 ghadi that's not what I see on my machine #2018-03-22 19:17:24 dominicm I think the script changed recently #2018-03-22 16:21:37 dominicm Looking at the script, I think it could be different to make these problems go away. Maybe I'll take a stab at tackling them. #2018-03-22 16:21:52 dominicm @alexmiller What is your process for testing the linux installer? #2018-03-23 15:28:57 borkdude I just tested the linux installer in a docker container openjdk:10 #2018-03-23 15:31:04 borkdude That worked. In openjdk:9 I got: Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty #2018-03-23 15:32:25 borkdude {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "1e6c71716a5e7b2e2f7d2f7d2b2a78262e78"}, :content ("[email protected]")}
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "45372a2a31057677232172737c23777d7671"}, :content ("[email protected]")} #2018-03-23 15:33:18 alexmiller @dominicm sorry for non-response above - I saw it but got distracted before answering and then the notifier was gone so forgot about it. :) I don’t test the linux installer very well (just in a vm) and would appreciate any help in making it better. #2018-03-23 16:15:59 dominicm Okay, I'll have a think. #2018-03-23 15:35:01 alexmiller @borkdude if you’re trying to compare times, make sure that you’re comparing cached classpath in both cases by running more than once #2018-03-23 15:35:21 alexmiller they mentioned that jvm startup times should be expected to be better in java 10 than java 9 #2018-03-23 15:35:46 borkdude I’m comparing 8 and 9 here with just vanilla clojure after it downloaded the bootstrap dependencies #2018-03-23 15:36:07 borkdude it seems to be a little bit better #2018-03-23 15:36:39 borkdude this is openjdk, no idea if there is any difference with the Oracle JDK #2018-03-23 15:36:45 borkdude not a JVM expert #2018-03-23 15:37:30 borkdude it’s very easy to spin up a Docker: docker run -it openjdk:10 /bin/bash and then do the clj linux installer #2018-03-23 16:06:25 tianshu how to do test with tools.deps? #2018-03-23 16:07:40 alexmiller What are you trying to test? #2018-03-23 16:08:19 tianshu I'm trying to use tools.deps instead of lein. I just want to run test like lein test. #2018-03-23 16:09:54 alexmiller Check out the test-runner project #2018-03-23 16:11:10 tianshu this is exactly what I'm looking for, thanks! #2018-03-23 16:11:50 alexmiller Other useful tools #2018-03-23 16:12:48 tianshu great #2018-03-23 16:21:31 tianshu when I use git/url, met an error
org.eclipse.jgit.api.errors.TransportException:
I just have get the :sha with git rev-parse HEAD. #2018-03-23 16:21:55 tianshu I'm confused what should I do? #2018-03-23 16:22:42 alexmiller @ghadi had a list of git connection issues in the back chat #2018-03-23 16:23:35 alexmiller In this case looks like maybe you’re using the ssh url when the https would be better #2018-03-23 16:23:54 ghadi https would work because that repo is public #2018-03-23 16:24:28 ghadi otherwise try ssh-keyscan -H >> ~/.ssh/known_hosts (I'm a bit surprised that https://github.com isn't in known_hosts -- perhaps you're containerized?) #2018-03-23 16:24:49 alexmiller I will update the test-runner read me to use https too #2018-03-23 16:25:14 alexmiller No reason for an ssh url there #2018-03-23 16:27:24 tianshu I wonder where tools.deps put the git dependencies? #2018-03-23 16:28:46 tianshu maybe I can find it with clj -Spath #2018-03-23 16:29:36 tianshu okay, found it. #2018-03-23 16:31:10 tianshu thanks a lot. #2018-03-23 17:00:57 dominicm I opened an issue for that @alexmiller #2018-03-23 17:58:37 alexmiller where? #2018-03-23 17:59:11 alexmiller nvm, I see it #2018-03-23 23:17:50 seancorfield @doglooksgood take a look at the deps.edn file in each of core.cache, core.memoize, and java.jdbc for examples of multiversion testing with clj and test-runner -- I recently converted all of those from Leiningen. #2018-03-24 02:01:41 tianshu @seancorfield It's very helpful. these project dose not have a resources folder, is there a resource-path in tools.deps or I should just use paths? #2018-03-24 03:02:21 seancorfield @doglooksgood the deps stuff is just about the class path. There's no distinction between the things that are on the class path. #2018-03-24 03:04:53 seancorfield If you use my boot-tools-deps then it has the idea of Boot resource path -- anything that would go in the uberjar -- and source path -- code that doesn't go in the uberjar. That puts deps paths into resource paths and extra-paths into source paths. #2018-03-24 10:19:01 dominicm Also, the deps.edn files should be somewhere in /etc I think too? Reaching out here for some community feedback. #2018-03-25 21:27:48 danielcompton @alexmiller did you hear what Java versions people were running that had issues with http://repo.clojars.org? #2018-03-26 00:03:40 alexmiller No #2018-03-26 11:52:06 mfikes An interesting aside to the above: Even though Java isn't required to install or use the Planck binary, homebrew doesn't yet make a distinction, so you transitively need Java via Clojure to install the binary. https://github.com/mfikes/planck/issues/695 #2018-03-26 12:20:43 alexmiller you can use :deps/root attribute for the relative subdir in the repo #2018-03-26 12:22:31 alexmiller {:git/url ""
:sha "b3d1d16d4915948a36b341f5648e4c9763628ef3"
:deps/root "figwheel-core"} #2018-03-26 12:26:02 roman01la should it work for Clojure 1.9? or do I need 1.10? #2018-03-26 12:26:21 alexmiller it declare the deps as needing only clojure 1.8 #2018-03-26 12:26:35 alexmiller but clojurescript 1.10.x #2018-03-26 12:41:27 roman01la I’m getting Error building classpath. nil when running clj with the following deps.edn
{:deps {figwheel-core {:deps/root "figwheel-core",
:git/url "",
:sha "b3d1d16d4915948a36b341f5648e4c9763628ef3"}}}
#2018-03-26 12:42:40 alexmiller I can’t reproduce that #2018-03-26 12:42:53 alexmiller is there a stack trace? #2018-03-26 12:43:21 alexmiller from clj -Sverbose, what version are you running? #2018-03-26 12:43:41 roman01la 1.9.0.358 #2018-03-26 12:43:49 alexmiller that’s latest #2018-03-26 12:44:04 alexmiller do you have anything in your ~/.clojure/deps.edn? #2018-03-26 12:45:02 roman01la There was deps, but I’ve edited it to be just {} before running #2018-03-26 12:45:19 alexmiller maybe also use -Sforce to make sure you’re not getting something from cache #2018-03-26 12:45:27 roman01la ok #2018-03-26 12:46:15 alexmiller oh interesting, that’s down in gitlibs. well, you could try cleaning your git cache in ~/.gitlibs #2018-03-26 12:46:28 alexmiller either selectively or by deleting the whole thing #2018-03-26 12:48:45 alexmiller that looks like it’s getting a nil url, which is weird #2018-03-26 12:55:05 roman01la Works now. Not sure how exactly, but after cleaning ~/.gitlibs it didn’t work first. Once I ran clj -Sforce it went fine and subsequent clj (without forcing) are ok now #2018-03-26 12:55:09 roman01la Thanks! #2018-03-26 13:07:43 alexmiller I suspect something bad in your cpcache #2018-03-26 15:47:02 mfikes :+1: I managed to get clj to run in the Ubuntu PPA build for Planck. The main trickery was really ensuring it stays in the sandbox, which is a little tighter than Homebrew—you don't even have a home directory, for instance. #2018-03-27 01:00:57 ghadi what a great article. #2018-03-27 03:12:06 richiardiandrea Yes #2018-03-27 16:20:44 dominicm I'm getting a weird issue. In Heroku, the clone step is failing for a https url with "authentication is required but no credentialsprovider has been registered". I can't think what the cause would be? #2018-03-27 16:20:55 dominicm Maybe it's something in ~/.ssh/config 😮 I don't know though. #2018-03-27 16:32:16 alexmiller this is a git dep? #2018-03-27 16:32:32 alexmiller with https url shouldn’t be using ssh at all #2018-03-27 16:33:23 alexmiller are you hitting a private repo via https? that’s not yet supported #2018-03-27 16:47:51 dominicm No, it's a public one. #2018-03-27 16:49:29 dominicm Oh yes, duh, of course ssh isn't engaged here at all. #2018-03-27 16:52:50 dominicm :extra-deps {
io.dominic/krei.alpha {:git/url ""
:sha "778ad4eb6130b7023204bf526b6f49d1429e4132"}}
Works locally, but not in heroku, it's definitely something contextual. #2018-03-27 16:56:34 dominicm Haha. I see the problem now. There's a typo in my username. I'm so terrible. Sorry for the noise. #2018-03-27 18:40:39 jeff.terrell For what it's worth, I learned recently that you can add custom ssh keys to Heroku, so that it can fetch deps from private git repos via SSH. (I had to do that because we were using a private npm package with github coordinates instead of npm registry coordinates.)
https://github.com/simon0191/custom-ssh-key-buildpack
(I realize you already solved your problem, @dominicm, it's just a useful capability to be aware of, so seemed worth sharing.) #2018-03-27 18:41:21 dominicm That's very useful, thank you #2018-03-27 21:51:35 richiardiandrea so that it can be called directly from clj #2018-03-27 21:51:45 richiardiandrea as in boot -m boot.main -param1 -param2 #2018-03-27 21:51:57 richiardiandrea ClojureScript does something similar at the moment and it is definitely a nice pattern #2018-03-27 21:53:20 richiardiandrea oh ok cool didn't read that thanks #2018-03-27 21:56:05 dominicm oh, that may be what I meant. #2018-03-27 21:56:18 dominicm clojureverse is down for me, so I was stabbing in the dark 😛 #2018-03-28 04:55:04 semperos When specifying :git/url in my deps.edn, I appear to have to use http:// rather than https://. When using https://, the initial message indicates that it's cloning anonymously, but then I get error messages and stack traces showing that it's trying to use {:tag :mailto:gitgithub.comgitgithub.com, :attrs nil, :content nil}. Using http:// does not present this problem and the repo is cloned anonymously as expected.
## clj details
clj -Sdescribe
{:version "1.9.0.358"
:config-files ["/usr/local/Cellar/clojure/1.9.0.358/deps.edn" "/Users/dgregoire/.clojure/deps.edn" "deps.edn" ]
:install-dir "/usr/local/Cellar/clojure/1.9.0.358"
:config-dir "/Users/dgregoire/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
## deps.edn entry for depstar (that fails)
com.healthfinch/depstar {:git/url ""
:sha "4aa7b35189693feebc7d7e4a180b8af0326c9164"}
## Error that occurs when it tries to check out depstar
Cloning:
Error building classpath. #2018-03-28 05:06:44 alexmiller seems weird. can you use -Sforce too to make sure you’re not seeing something in cache? #2018-03-28 05:17:07 semperos -Sforce shows same behavior; deleted ~/.gitlibs and local .cpcache as well, no change #2018-03-28 05:18:11 seancorfield @semperos {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "8aede3fecaede3fee2ffe8a4e9e5e7"}, :content ("[email protected]")} does not indicate an https: URL -- that sounds like you've used an SSH URL instead. #2018-03-28 05:20:03 semperos could be that it's late and I'm missing something obvious, but the :git/url in my deps.edn (pasted above) is "" #2018-03-28 05:21:46 seancorfield This worked for me {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "d7a4b2b6b997a4b2b6b9faafa7a4"}, :content ("[email protected]")} #2018-03-28 05:22:14 seancorfield So try a later SHA. #2018-03-28 05:22:47 semperos clj -Sdeps '{:deps {com.healthfinch/depstar {:git/url "" :sha "68f3a04f75c988fa032974e8c600c9086812e9ef"}}}'
Cloning:
Error building classpath. #2018-03-28 05:23:10 semperos not limited to this repo, it's happening for others #2018-03-28 05:23:14 seancorfield Are you on the latest clj / tools.deps ? #2018-03-28 05:23:47 semperos I pasted my info above, I believe I am; my $HOME/.clojure/deps.edn is blank and my /usr/local one is:
{
:paths ["src"]
:deps {
org.clojure/clojure {:mvn/version "1.9.0"}
}
:aliases {
:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.5.417"}}}
:test {:extra-paths ["test"]}
}
:mvn/repos {
"central" {:url ""}
"clojars" {:url ""}
}
}
#2018-03-28 05:24:07 seancorfield {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "b7c4d2d6d9f7c4d2d6d99acfc7c4"}, :content ("[email protected]")} #2018-03-28 05:24:49 semperos I deleted the cache in the local folder, I'll try clearing out the other cache dirs // no dice #2018-03-28 05:26:09 semperos clj -Sverbose
version = 1.9.0.358
install_dir = /usr/local/Cellar/clojure/1.9.0.358
config_dir = /Users/dgregoire/.clojure
config_paths = /usr/local/Cellar/clojure/1.9.0.358/deps.edn /Users/dgregoire/.clojure/deps.edn deps.edn
cache_dir = /Users/dgregoire/.clojure/.cpcache
cp_file = /Users/dgregoire/.clojure/.cpcache/2178403036.cp
Refreshing classpath
Clojure 1.9.0
user=>
#2018-03-28 05:29:51 seancorfield I just tried it on OS X like you Tue Mar 27 22:28:33
(sean)-(jobs:0)-(~/clojure/semperos)
(! 504)-> clj -Sdeps '{:deps {com.healthfinch/depstar {:git/url "" :sha "68f3
04f75c988fa032974e8c600c9086812e9ef"}}}'
Cloning:
Checking out: at 68f3a04f75c988fa032974e8c600c9086812e9ef
Clojure 1.9.0
user=> ^D
Tue Mar 27 22:29:20
(sean)-(jobs:0)-(~/clojure/semperos)
(! 505)-> #2018-03-28 05:30:23 seancorfield (sorry for the weird line breaks there -- I'm ssh'ing into my Mac from Ubuntu on my Windows 10 laptop!) #2018-03-28 05:31:26 semperos thanks for giving it a try; glad to know I'm the exception in this regard 🙂 #2018-03-28 05:34:31 semperos picard-facepalm I apologize for the spam, I have an entry in my global ~/.gitconfig that was the culprit #2018-03-28 05:35:39 seancorfield Oh, interesting... what was it? #2018-03-28 05:35:45 semperos [url "
#2018-03-28 05:37:14 semperos if I do straight git clone things work as expected #2018-03-28 05:37:20 seancorfield Ah... that's good to know... #2018-03-28 14:28:16 ghadi @semperos JGit is too clever #2018-03-28 14:28:31 semperos my thoughts too 🙂 #2018-03-28 18:10:57 stephenmhopper Hi, I have two questions. (1) Is there a way to make options required for tools.cli and (2) why doesn't the tools.cli repo have an issues tab? https://github.com/clojure/tools.cli #2018-03-28 18:12:19 ghadi Clojure projects's issues are managed through JIRA, not Github -- see CONTRIBUTING.md #2018-03-28 18:13:12 ghadi bug tracker ^ #2018-03-28 18:14:12 stephenmhopper Ah, I guess I didn't realize that JIRA was used for anything not in the core Clojure releases. Also, I think I might have answered my own question: https://dev.clojure.org/jira/browse/TCLI-12 #2018-03-28 18:35:46 seancorfield @stephenmhopper Is the :missing approach sufficient for you? (and, BTW, tools.cli is not tools.deps / clj -- that seems to be a common confusion nowadays) #2018-03-28 18:39:36 stephenmhopper @seancorfield Yeah, :missing seems to be working for me. And yeah, I wasn't sure if this was the right channel or not. #clojure seemed too broad. Perhaps setting the topic for the channel would prevent confusion from folks in the future. #2018-03-28 18:42:04 seancorfield (The purpose mentions tools.dep api but that only shows when you have the Channel Details panel open) #2018-03-28 18:42:29 stephenmhopper Cool. Thank you! #2018-03-28 18:43:58 seancorfield I've also had folks open JIRA issues against tools.cli thinking it's the clj/`clojure` command line tooling 🙂 #2018-03-28 18:44:20 seancorfield I maintain tools.cli so I'm happy to field any questions you have about it. #2018-04-01 22:28:48 seancorfield @dominicm What is the final argument there? It looks like a SHA but doesn't match anything else? What am I missing? #2018-04-02 05:28:42 dominicm It's the latest version sha. The sha of injector is newer than the code for it. They can probably match (I could potentially even automatically pick it up using horrible hacks) #2018-04-02 02:22:32 ehashman Not quite a tools.deps question, but I'm not sure of a better place to ask---does anyone have any tips on debugging why an AOT-compiled app would be re-JIT compiling when run? #2018-04-02 02:23:16 ehashman I have not looked inside the jar to check the timestamps, but I don't currently have a reason to believe something modified the source code timestamps to be different than the compiled artifacts #2018-04-02 02:27:52 alexmiller There is a bug in the maven shade plugin that doesn’t preserve time stamps but not many people use that #2018-04-02 03:01:26 mikerod That is a related discussion I had on this quite a while back #2018-04-02 03:02:41 mikerod When I was using maven shade plugin, my solution was to use maven exec plugin to open the shaded jar back up, bump all clj source time stamps back to something in the past and repack the jar. Did that in a script. Hacky #2018-04-02 03:03:35 mikerod You could also do the workaround there about not including source clj files #2018-04-02 03:05:22 kidpollo are there any guides to using deps + cider? #2018-04-02 03:05:47 kidpollo I am having trouble getting it going with refactor-nrepl #2018-04-02 03:50:06 seancorfield @kidpollo Did you try asking in #cider ? #2018-04-02 03:50:35 seancorfield (I haven't tried it -- I only recently heard that CIDER supports deps.edn) #2018-04-02 04:41:44 kidpollo it does but cant get the refactoring tooling to work #2018-04-02 04:53:24 seancorfield Hmm, I'm not sure why deps vs lein/boot would affect the refactoring tool...? #2018-04-02 05:41:04 dominicm cider-jack-in, last I checked, hardcoded the deps list. It hadn't yet started using plugins properly. #2018-04-02 20:26:13 dominicm https://github.com/juxt/pack.alpha#auto-add-to-project now supports updates when an alias for pack exists, regardless of name.
I really like the approach that I'm taking here, I think it has legs. I could even see an alternative version of -Sresolve-tags based on this, which doesn't alter your indentation at all. That would be quite valuable, given how varied the indentation preferences are in the community. #2018-04-05 18:18:45 alexmiller note that lein-tools-deps is using a much older version of tools.deps.alpha than master #2018-04-05 18:22:18 alexmiller I’m not sure what the problem is that you’re seeing, maybe something to do with loading the right extension namespaces first (some of that is now done for you in current tda but did not used to be) #2018-04-05 19:16:05 phreed Apparently the-locator is nil. #2018-04-05 19:23:06 alexmiller Well that should never be :) #2018-04-05 19:24:04 alexmiller What’s up with the other deps - do those match what tda is using? #2018-04-05 19:24:57 alexmiller Those seem to be the old aether libs not maven-resolver #2018-04-05 19:26:05 phreed What is tda? #2018-04-05 19:26:40 phreed ...and what other deps? #2018-04-05 19:28:07 phreed tools.deps.alpha #2018-04-05 19:30:31 phreed You mean [org.eclipse.aether/aether-api "0.9.0.M2"] ? #2018-04-05 19:30:53 phreed I put those in to try to fix the issue, it did not change anything. #2018-04-05 20:16:12 alexmiller As you can see locator is a delay instance, so it should never be nil, even if it failed to connect #2018-04-05 20:29:48 phreed Right, I moved things around so I could see what was going on better and now I get...
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: No matching method: newServiceLocator, compiling:(lein_tools_deps/plugin.clj:52:3)
on
(def locator
(MavenRepositorySystemUtils/newServiceLocator
(.addService RepositoryConnectorFactory BasicRepositoryConnectorFactory)
(.addService TransporterFactory FileTransporterFactory)
(.addService TransporterFactory HttpTransporterFactory)
(.addService TransporterFactory WagonTransporterFactory)
(.setService WagonProvider CustomProvider))) #2018-04-05 20:42:05 phreed And, when I use clojure.reflect it shows
| :name | :return-type | :declaring-class | :parameter-types | :exception-types | :flags |
|-------------------+---------------------------------------------------+-----------------------------------------------------------------+------------------+------------------+--------------------|
| newServiceLocator | org.eclipse.aether.impl.DefaultServiceLocator | org.apache.maven.repository.internal.MavenRepositorySystemUtils | [] | [] | #{:public :static} |
| newSession | org.eclipse.aether.DefaultRepositorySystemSession | org.apache.maven.repository.internal.MavenRepositorySystemUtils | [] | [] | #{:public :static} | #2018-04-05 20:44:40 phreed I am an idiot. I removed the doto as well as the delay. #2018-04-05 20:48:42 richiardiandrea hello folks, I was wondering if somebody else is working on updates to the linux installer by any chance (e.g., use install) ? #2018-04-05 21:35:06 alexmiller There are some outstanding patches that I intend to apply for next #2018-04-05 22:15:14 dominicm I wrote a patch for install after we had issues on training #2018-04-05 23:49:06 richiardiandrea cool thanks for sharing, meaning to have a look at that after we had problems too here #2018-04-05 21:40:03 phreed I upgraded leiningen to 2.8.1 (from 2.7.x) and the problem resolved itself. Maybe a transitive dependency on a maven library. Thanks. #2018-04-05 21:54:01 alexmiller Yikes #2018-04-05 22:55:25 seancorfield @phreed FYI, if you use triple back-ticks around code, it's much easier to read in Slack like this
#2018-04-08 06:44:25 seancorfield FYI [seancorfield/boot-tools-deps "0.4.4"] just went to Clojars: it bumps the tools.deps.alpha dependency to 0.5.417 to address the slf4j logger warning. #2018-04-08 16:15:43 mfikes I have a plk script that composes clojure (for the purposes of classpath construction) with planck. This is working out fairly nicely, but there is a hack I needed to do in order to support :main-opts, by parsing the output of -Sverbose 😱
It makes me wonder if, akin to -Spath, a -Smain-opts would be generally useful.
(The fragile hack is at the bottom of https://gist.github.com/mfikes/bbf8a9a5b9ed6ecac15a3bb75d2dc8b4 if curious.) #2018-04-08 16:16:36 mfikes (Or alternatively, more info in -Sdescribe could lead to a solution.) #2018-04-08 18:07:01 alexmiller Happy to consider additions to -Sdescribe - file a ticket if so #2018-04-08 19:32:12 mfikes At the very bottom of /usr/local/bin/clojure, does it make sense to have the script do
exec "$JAVA_CMD" ...
in order to have the shell script launching the Clojure process terminate (or be replaced). I'm asking because TBH I don't fully appreciate the consequences and wondering if anyone familiar with this aspect of Unix might know. #2018-04-08 22:25:16 tianshu is it possible to get the paths in program when using tools.deps. I try to write a figwheel.clj build script, but don't want hardcode the paths. #2018-04-08 23:24:04 alexmiller @doglooksgood could you explain more about what paths you mean? #2018-04-08 23:25:09 alexmiller @mfikes et al - I have deprecated the INST jira project, please use TDEPS for anything installer / clj related. Since I work on both of these in tandem, it’s just easier for me to manage and prioritize one bucket instead of two. And if you find a link to the INST one, let me know so I can update it. #2018-04-08 23:25:45 mfikes Will do! #2018-04-08 23:26:10 tianshu @alexmiller I want the :paths in deps.edn. #2018-04-08 23:28:06 alexmiller generally speaking, I would say yes. but the answer is somewhat subtle. #2018-04-08 23:32:07 alexmiller there are really three different levels at work here:
1) clj command line script (bash) - fully environment aware with knowledge of multiple sources for deps.edn and caching concerns. The script knows how to find all deps.edn sources and invoke make-classpath
2) make-classpath program (Clojure) - a small program that knows how to read and combine things it is told by #1 to use tools.deps to build a classpath
3) tools.deps resolve-deps / make-classpath - know how to do dependency resolution etc #2018-04-08 23:33:15 alexmiller to find the same :paths that clj will use, you need at least the top two levels (and possibly the third, if you want to consider the modifications from :extra-paths in aliases) #2018-04-08 23:35:46 alexmiller but you might be able to take shorter paths to do things that are usually enough - it depends what other constraints/assumptions you can rely on and how close to clj you need to be. #2018-04-08 23:37:08 alexmiller for example, if you know your local project.clj always has the paths in it and that’s all you want, then you could just literally read the file as edn and extract the :paths key. #2018-04-08 23:37:20 alexmiller and you don’t even need tools.deps to do that #2018-04-08 23:38:23 tianshu I my case, I write a .clj file which is a script file to launch figwheel build, and this require a source-path that contains .cljs files. I want to specify source-path with aliases in tools.deps #2018-04-08 23:40:51 alexmiller there’s no way to self refer to the paths via a magic variable or anything like that #2018-04-08 23:43:09 tianshu once the clj or clojure started, the paths will never change? #2018-04-08 23:47:14 tianshu if it is, provide apis or variables can make something easier, like finding the paths or deps. so some general purpose scripts can be write and reuse. I think this can help people use their own project to customize their workflow. #2018-04-08 23:52:54 alexmiller as described above, there are a lot of nuances to how clj and tools.deps works #2018-04-08 23:53:33 alexmiller this tool is designed to launch programs prepared with certain dependencies on the classpath #2018-04-08 23:54:41 alexmiller perhaps a better way to ask the question is - can you write a program that can be launched by this tool that will output the desired information? #2018-04-08 23:55:52 alexmiller like clj -e '(System/getProperty "java.class.path")'
#2018-04-08 23:55:53 tianshu yes, that is.😶 sorry for my bad question. #2018-04-08 23:56:09 alexmiller not a bad question, just trying to rephrase to alignment with the goal of the tool #2018-04-08 23:57:12 alexmiller is the output of the command above the answer you seek? or at least the input? that could easily be filtered to sub paths of the current directory #2018-04-08 23:58:51 tianshu yes, I think this is what I seek! #2018-04-08 23:59:09 tianshu such a simple way:joy: #2018-04-09 00:00:15 tianshu thanks very much! #2018-04-09 16:20:47 drewverlee I think it would be really cool to write/blog/livecast someone translating a project.clj to a deps.edn. I mainly think this is cool because im having to do it right now and its very education, by which i mean, i have lots of things I dont yet understand 🙂 #2018-04-09 16:24:40 dominicm Oh, I did it by regex #2018-04-09 16:25:05 dominicm :%s/\[\(\S\+\) \(".*"\)\]/\1 {:mvn\/version \2} is how I did it in vim. #2018-04-09 16:25:22 dominicm Then I also ran :%s/\[\(\S\+\) \(".*"\)\ \(:exclusions \[.*\]\)]/\1 {:mvn\/version \2\r\3} #2018-04-09 16:25:52 drewverlee is it that mechanical of a process? Its possible this hinges on my understanding of what deps.edn does vs what lein project.cljs do #2018-04-09 16:26:17 dominicm @drewverlee deps.edn only specifies dependencies. So that's the only part that really converts over from project.clj. #2018-04-09 16:26:21 drewverlee I guess your right, the only thing i have to transfer is the deps #2018-04-09 16:26:23 drewverlee yep #2018-04-09 16:27:42 drewverlee @dominicm
out of curiosity, for those trying to use deps.edn, what are they using for
1) profiles
2) build stuff
3) deployment
I’m guessing boot can still handle these responsibilities #2018-04-09 16:27:58 dominicm There are tools, like pack.alpha which add uberjars on top. #2018-04-09 16:28:54 dominicm (although it's out of date, see the pack readme!) #2018-04-09 16:36:05 alexmiller aliases can serve some of the same purposes as profiles. pack.alpha and depstar are two build options. #2018-04-09 16:37:35 alexmiller if you have consumers use git deps, you can skip build and deploy altogether :) (I am somewhat joking - this is not feasible for a number of cases) #2018-04-09 17:13:48 dominicm @alexmiller I've been thinking about krei, there's an API for performing java complication in the jdk, so in theory if I search for an edn file describing how to build from source, it could be feasible. #2018-04-09 17:14:15 dominicm Having git dependencies is a lot like working on a mono repo, which is the pain point krei was designed to solve. #2018-04-09 17:15:07 dominicm I think it could work, in theory. Haven't tried that hard yet though. #2018-04-09 17:24:38 alexmiller What’s krei? #2018-04-09 17:32:53 dominicm @alexmiller Krei is a in-repl build tool. It scans the classpath for krei-file.edn files, and then builds sources on the classpath from that.
Currently it supports cljs and sass, because that's what I need for the itch I'm scratching. But I think it could work for java too. Protobuf is probably important to target also. #2018-04-09 17:33:14 alexmiller gotcha #2018-04-09 17:36:22 dominicm Maybe I should split into manifests and try to parse information from pom files? Dunno. #2018-04-09 17:43:34 alexmiller well I can tell you that parsing info from pom files suuuuucks #2018-04-09 17:45:50 dominicm Only using edn it is! #2018-04-09 19:19:20 seancorfield So :profiles => various :aliases, "build" can either use tooling built on top of deps.edn (for example, I use boot-tools-deps with Boot on top of deps.edn and there's a Leiningen plugin, which is very much a work-in-progress), or it can use the JAR builders that are listed on that wiki page... #2018-04-11 15:01:46 bhauman as folks migrate to tools-deps are people still using a "resources" directory to indicate static resources? #2018-04-11 15:02:21 alexmiller probably so #2018-04-11 15:06:36 bhauman I'm trying to come up with reasonable initial instructions for where people should place their index.html #2018-04-11 15:07:12 bhauman so I'll keep rolling with resources/public/index.html #2018-04-11 15:08:53 bhauman also I was surprised that if you place "resources" in :paths that "resources" gets appended to the classpaths of all the :local/root's #2018-04-11 15:29:11 dominicm that sounds like a bug. #2018-04-11 15:36:48 alexmiller that is a bug that has been mentioned before. possibly no one logged it as I had forgotten about it. #2018-04-11 15:38:01 alexmiller definitely will fix #2018-04-11 16:03:17 bhauman oh cool, I was worried that it was on purpose #2018-04-11 16:09:21 alexmiller no, definitely not #2018-04-12 22:45:19 hagmonk I spent some time trying to make AOT for my app work with tools.deps, finally watched dtrace for a bit and figured out the magic :) #2018-04-12 22:45:52 hagmonk In this case I'm also pulling in rebel-readline and clj-async-profiler:
clj -Sdeps "{:deps {com.bhauman/rebel-readline {:mvn/version \"0.1.2\"} com.clojure-goes-fast/clj-async-profiler {:mvn/version \"0.1.3-SNAPSHOT\"}} :paths [\"classes\" \"src\"]}" -m rebel-readline.main
#2018-04-12 22:46:27 hagmonk Oh, and one must make the classes folder in advance too #2018-04-12 23:10:14 hagmonk I found some deps on libs pulled from our corporate artifactory instance were missing, seemingly because they have the scope "runtime" set #2018-04-12 23:33:05 alexmiller That’s intentional but still thinking about whether to include runtime #2018-04-13 00:12:09 seancorfield What does a runtime scope mean for Maven? #2018-04-13 00:33:50 alexmiller It’s for running things :) #2018-04-13 00:34:08 alexmiller But not needed to compile #2018-04-13 03:21:14 hagmonk Hum, well, that's an interesting distinction #2018-04-13 03:22:51 hagmonk Is the classpath assembled for a deps.edn based project intended to get it only as far as compilation? I hit this problem through a transient dependency, and I actually had to use lein deps :tree to discover the problem, because tools.deps.alpha completely filtered out the existence of the transient dependency #2018-04-13 03:39:51 alexmiller yeah, there’s a ticket for this already #2018-04-13 03:40:39 alexmiller feel free to add your vote #2018-04-13 03:49:33 hagmonk will do sir, thanks #2018-04-13 06:52:11 seancorfield @richiardiandrea made me do it! clj -Sdeps '{:deps {seancorfield/clj-new {:git/url "" :sha "653441d86834160124a94065c4d92a4a9ad12554"}}}' -m clj-new.create app myapp/clj-test The beginnings of a clj-based new project template generator... #2018-04-13 13:16:53 john So, I can just do clj -m clj-new.create figwheel myapp and it'll bring in the leiningen template and build it? #2018-04-13 13:34:35 john When I do clj -Sdeps '{:deps {seancorfield/clj-new {:git/url "" :sha "653441d86834160124a94065c4d92a4a9ad12554"}}}' -m clj-new.create figwheel figfig #2018-04-13 13:35:22 john I get
Failed with: Failed to read artifact descriptor for figwheel:lein-template:jar:RELEASE
Exception in thread "main" clojure.lang.ExceptionInfo: Could not load artifact for template: figwheel
Tried coordinates:
[figwheel/boot-template "RELEASE"]
[figwheel/lein-template "RELEASE"] {}
at clojure.core$ex_info.invokeStatic(core.clj:4754)
at clojure.core$ex_info.invoke(core.clj:4754)
...
#2018-04-13 14:50:49 richiardiandrea All in clj ;) #2018-04-13 15:30:03 seancorfield As noted, work in progress. Needs more work before it can do more than the built-in templates. But, yes, the plan is that will work. #2018-04-13 15:30:31 john niiiiiice! #2018-04-13 15:33:16 seancorfield I think that fails because I haven't set up the full clj environment for t.d.a. yet (in particular the repos). #2018-04-13 20:18:47 seancorfield @U050PJ2EU OK, clj -Sdeps '{:deps {seancorfield/clj-new {:git/url "" :sha "a87bdc0347fe1523e8d85283d94b06f5dd68b2cb"}}}' -m clj-new.create figwheel figfig This should work now! #2018-04-13 20:19:16 john Hahah! #2018-04-13 20:20:05 john tworks! #2018-04-13 20:20:33 john this is newsworthy #2018-04-13 20:20:55 john we'll have to whip up some clj/cljs.main templates 🙂 #2018-04-13 20:24:45 seancorfield I plan to add clj-template support soon, so we can have clj templates that produce deps.edn-based projects. #2018-04-13 20:46:06 richiardiandrea let me try macchiato #2018-04-13 20:48:29 richiardiandrea works! awesome 😄 #2018-04-13 06:53:17 seancorfield (it currently generates boot templates by default -- but it should be able to generate any Leiningen template too) #2018-04-13 06:54:12 dominicm Lol, that's great. #2018-04-13 06:54:30 seancorfield He made me itch. I had to scratch it. #2018-04-13 06:55:21 dominicm I spent far too long last night writing an ANSI parser, I feel the sentiment quite hard right now :) #2018-04-13 14:52:18 alexmiller someone dropped a docker command to spin a linux box for testing clj installs here a couple weeks ago - anyone have that handy? #2018-04-13 14:54:19 dominicm I wouldn't use that for testing #2018-04-13 14:55:17 dominicm Docker runs as root within the container, you miss a key aspect of testing that way #2018-04-13 14:55:39 alexmiller I wasn’t only going to only use it #2018-04-13 15:50:46 mkvlr > docker run -ti openjdk:8 /bin/bash #2018-04-13 15:50:51 hagmonk @dominicm that's true by default but it's trivially disabled #2018-04-13 15:54:59 mkvlr apt-get update
apt-get install -y rlwrap curl
export CLOJURE_VERSION=1.9.0.341 && curl -O && /bin/bash linux-install-${CLOJURE_VERSION}.sh #2018-04-13 15:55:04 hagmonk although for the purposes of installing clojure, it looks like the getting started guide recommends root anyway #2018-04-13 15:55:22 mkvlr @alexmiller these two things should do or do you want a built image? #2018-04-13 15:57:31 dominicm @hagmonk running it is the difficulty #2018-04-13 15:58:04 hagmonk running it? #2018-04-13 17:04:51 dominicm Running the script as non-root is the important part, to ensure that OS permissions are correctly set #2018-04-13 15:58:48 hagmonk docker run -ti openjdk:8 bash -c "$(curl ) && clojure" #2018-04-13 16:08:55 hagmonk Now with @bhauman power up mode: docker run -ti openjdk:8 bash -c "$(curl ) && clojure -Sdeps \"{:deps {com.bhauman/rebel-readline {:mvn/version \\\"0.1.2\\\"}}}\" -m rebel-readline.main" #2018-04-13 16:09:54 hagmonk I have a docker container internally that already has a .cpcache for fast rebel readline startup in a container :) Useful for when I need a totally clean slate #2018-04-13 16:33:08 alexmiller do you think the prefix dir should be created if it doesn’t exist? #2018-04-13 16:38:28 alexmiller with the TDEPS-65 patch for the linux install, it will be created if it doesn’t exist #2018-04-13 16:38:47 alexmiller I think I’m ok with that behavior #2018-04-13 17:03:12 dominicm See no reason not to add a guard if we want to ensure that the prefix exists already. I suppose it's keeping with other tools to not create leading directories by default. I ignored that comment because I was fairly certain that the current behavior didn't follow that. #2018-04-13 17:27:30 hagmonk I'd actually argue, just from memory and experience, that directories should be created if it they don't already exist. I can't recall the last time I supplied any directory or prefix argument to a tool and then thought "shoot, I have to manually make that directory" #2018-04-13 17:27:50 hagmonk But that's just my own experience #2018-04-13 17:29:29 alexmiller this is mostly an issue for when you are supplying the prefix dir #2018-04-13 17:29:52 hagmonk right, I'd expect that if I supply the prefix dir, it will be created if it doesn't exist #2018-04-13 17:29:58 alexmiller yeah #2018-04-13 17:30:17 hagmonk I'm trying to think of a case where I've done something similar recently #2018-04-13 17:31:09 hagmonk I guess maybe the classic case is ./configure --prefix=/usr/local/whatever #2018-04-13 17:31:18 hagmonk when using autotools #2018-04-13 17:32:27 hagmonk I checked the nearest autotools generated configure script … #2018-04-13 17:33:32 hagmonk they definitely mkdir -p, although they do it in the most gloriously overwrought tradition of GNU autotools:
# as_fn_mkdir_p
# -------------
# Create "$as_dir" as a directory, including parents if necessary.
as_fn_mkdir_p ()
{
case $as_dir in #(
-*) as_dir=./$as_dir;;
esac
test -d "$as_dir" || eval $as_mkdir_p || {
as_dirs=
while :; do
case $as_dir in #(
*\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'(
*) as_qdir=$as_dir;;
esac
as_dirs="'$as_qdir' $as_dirs"
as_dir=`$as_dirname -- "$as_dir" ||
$as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \
X"$as_dir" : 'X\(//\)[^/]' \| \
X"$as_dir" : 'X\(//\)$' \| \
X"$as_dir" : 'X\(/\)' \| . 2>/dev/null ||
$as_echo X"$as_dir" |
sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{
s//\1/
q
}
/^X\(\/\/\)[^/].*/{
s//\1/
q
}
/^X\(\/\/\)$/{
s//\1/
q
}
/^X\(\/\).*/{
s//\1/
q
}
s/.*/./; q'`
test -d "$as_dir" && break
done
test -z "$as_dirs" || eval "mkdir $as_dirs"
} || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir"
} # as_fn_mkdir_p
#2018-04-13 17:33:48 hagmonk eww sorry, I thought that would collapse #2018-04-13 18:37:49 seancorfield Question about classloaders -- when I run code with clj -m ... should I be able to get at a modifiable classloader? #2018-04-13 18:46:46 seancorfield It seems that when you're in the REPL, you have a DynamicClassLoader from Clojure, but when running via -m you only have sun.misc.Launcher$AppClassLoader and sun.misc.Launcher$ExtClassLoader neither of which are modifiable... #2018-04-13 18:57:38 dominicm The goal should be to act like a normal Linux program. Regardless of whether we expect those behaviours ourselves. #2018-04-13 19:04:23 seancorfield Well, to dynamically load Leiningen templates at runtime, I need a modifiable classloader... for now I've made one (`clojure.lang.RT/makeClassLoader`) and stuffed it back into the current thread. #2018-04-13 19:46:31 seancorfield Anyone happen to know what package org.sonatype.aether.collection.DependencyGraphTransformer is part of? When I try to load leiningen-core via t.d.a. this doesn't seem to get loaded... #2018-04-13 19:56:47 seancorfield NM, figured it out. #2018-04-13 20:23:08 john our diabolical plans are almost in place #2018-04-13 20:23:26 seancorfield Qualified project names do not work (they don't work in boot-new either). #2018-04-13 20:24:40 john this is super sweet though, thanks Sean! #2018-04-13 20:25:14 seancorfield I'll work on it more over the weekend and make an official announcement on Monday I expect. #2018-04-13 20:54:29 richiardiandrea Don't have to say how cool this is, no more lein or boot for template generation is a boon for lumo projects #2018-04-13 21:15:19 alexmiller new tools.deps.alpha 0.5.425 is out. new installers working their way through the meat grinder #2018-04-13 21:16:28 alexmiller mostly just bug fix releases for both, I’ll do a ggroups announcement when brew is done #2018-04-13 21:19:53 seancorfield I'll add it to the tools page after it's had some polish this weekend. There are a few issues (opened on GH!) that need addressing before it's ready for real world users! 🙂 #2018-04-13 21:42:08 john s/lein new/clj -A:new/ #2018-04-13 22:00:12 seancorfield clj -A:new right? 🙂 #2018-04-13 22:00:33 seancorfield (or does it support omitting the leading : now?) #2018-04-13 22:02:24 richiardiandrea I have put it directly in my :deps in ~/.clojure/deps.edn #2018-04-13 22:43:06 john ma bad #2018-04-13 22:44:16 john so what's it look like? the -m ... line goes in extra-args right? #2018-04-13 22:44:57 john ah, main-opts #2018-04-13 22:55:23 john Yeah, this works for me:
...
:aliases
{:new {:main-opts ["-m" "clj-new.create"]
:extra-deps
{seancorfield/clj-new {:git/url ""
:sha "a87bdc0347fe1523e8d85283d94b06f5dd68b2cb"}}}
...
#2018-04-16 17:01:03 timgilbert Adding this magic entry to ~/.clojure/deps.edn, or a project deps.edn, seems like something that could be written as a useful tool, similar to npm install or yarn add or whatnot - typing all of this info is slightly too annoying right now for use in shell scripts / one-liners IMHO #2018-04-16 17:04:17 dominicm I have this in pack.alpha #2018-04-16 17:04:25 dominicm Pack can inject itself for you. #2018-04-16 17:04:46 dominicm I'm thinking it might become a general injector eventually which you can then do clj -A:user/inject new to inject into deps.edn #2018-04-13 22:57:09 john then just do clj -A:new figwheel figthing #2018-04-13 23:03:22 alexmiller I intentionally wrote the code to be a little lax on that leading colon so it’s optional #2018-04-13 23:38:06 seancorfield The built in templates now generate projects with deps.edn files $ clj -A:new app myapp
$ cd myapp
$ clj -m myapp.core
Hello World!
$ clj -A:test:runner
...FIXME test failure...
#2018-04-14 00:24:37 seancorfield OK, something we've all wanted ever since Leiningen did the .core thing... clj-new SHA "f6fcc24bfa5d77167ff826990cd2c9c65eed4fed" prohibits single-segment or unqualified project names! #2018-04-14 00:25:12 seancorfield clj -A:new app foo/bar or clj -A:new app foo.bar #2018-04-14 00:25:20 seancorfield And, with that, I'm out for the day... #2018-04-14 00:45:50 richiardiandrea That's great thanks a lot Sean! #2018-04-14 07:56:59 seancorfield (rewriting the Boot generator stuff will probably be tomorrow/Sunday at this point) #2018-04-14 13:23:30 john So what would be the pros and cons of moving some projects off private mavens completely and just moving to private git shas? #2018-04-14 13:24:12 john or private git urls #2018-04-14 13:24:38 john seems like you're duplicating your deps anyway, if you've got your snapshot in github #2018-04-14 13:25:42 john While also just copying them to some yaven... #2018-04-14 13:28:39 john in other words, unless I'm providing to some maven consumer that can't use clj, why use maven at all now? #2018-04-14 13:33:46 john hmm, I wonder if a lot of clojure libs somehow depend on things living on maven... #2018-04-14 15:08:41 dominicm @john None I've run into depend on things being from maven. #2018-04-14 15:11:21 john @richiardiandrea @dominicm that's what I was thinking #2018-04-14 15:12:11 john @mfikes hmm. Does that just mean that some cljs things won't be cached on first pull of the dep? #2018-04-14 15:12:24 john But they still cache after that, right? #2018-04-14 15:13:42 john or, if someone is using :aot-cache true then it'll break with git deps, currently? #2018-04-14 15:14:23 dominicm It won't break. They just won't be candidates for caching. #2018-04-14 15:15:04 mfikes @john It just means that Git deps behave like things did before :aot-cache existed: They do still cache locally on first build. #2018-04-14 15:15:18 john rgr #2018-04-14 15:17:37 john so if you have a project where you're waiting to migrate to clj tool, because of the ongoing work on private maven repos, you might be able to just skip all that and depend on the git dep directly, right? #2018-04-14 15:18:48 dominicm As long as you control the private project you're interested in, yes. #2018-04-14 15:18:53 john Obviously I won't fully know until I try migrating a whole project over, but I'm just doing a sanity check first #2018-04-14 15:18:57 dominicm True for internal projects, false for datomic 🙂 #2018-04-14 15:19:08 john right #2018-04-14 21:16:38 seancorfield This was something that I noticed as I worked on clj-new -- I forked it from boot-new so at first it still had a build.boot, which was the only place where a version number was stored, and was the only place that had anything to do with building artifacts and deploying them (to Clojars). And I was suddenly "Oh, wait! I don't need a Clojars badge on this project, or version numbers, or artifact coordinates... 👀 " #2018-04-14 21:19:02 mfikes In ClojureScript there was a similar issue where it normally produces a version number as part of its build process, which caused a little grief with trying to use ClojureScript as a "buildless" git dep. But that has been resolved (parts of the system have been changed to cope without a version number). #2018-04-14 21:24:45 seancorfield @mfikes Some Leiningen templates expect to be able to get at Leiningen's version number, which it assumes is in META-INF/maven/leiningen/leiningen/pom.properties -- which is why both boot-new and clj-new have that tree structure in src. #2018-04-14 21:25:17 dominicm Would be interesting to know "your" version (as defined in the deps.edn) somehow. I guess it could be set by clj using system properties when starting a jvm... 🤔 #2018-04-14 21:25:58 mfikes In ClojureScript's case the solution we went with was to derive a fake version number from the hashes of all of the code in the source tree #2018-04-14 21:26:17 dominicm That gives me 2 use cases for contextual information now: tools.namespace could use it for reloading, and to grab a common piece of build information: version. #2018-04-14 21:27:01 dominicm I suppose it would need to be burned into the resulting UberJARs... but I can take of that 🙂 #2018-04-14 21:26:21 seancorfield And also the leiningen.release namespace. I tried to incorporate Leiningen 2.8.1 but it has a namespace with a def that slurps one of its own files when you load it(!), so I stuck with 2.7.1. Looks like it'll be "fixed" in 2.8.2. #2018-04-14 21:29:48 mfikes My takeaway: Anything that needs to be "built" can be a hinderance for just using code directly as a dep. #2018-04-14 21:30:05 mfikes (Including something as simple as a version number.) #2018-04-14 21:47:36 hagmonk hey this is germane to what I'm working on #2018-04-14 21:48:06 hagmonk I started patching depstar to support building thin jars and producing a folder of jar artifacts #2018-04-14 21:49:02 hagmonk for the purposes of building a kafka connector (which requires a thin jar + deps in a dir), and for separating deps into a separate docker file layer for speedier builds #2018-04-14 21:50:13 hagmonk question I'm facing at the moment: should things like version numbers or aliases that configure how to build the jar be in deps.edn, or kept exclusively as command line arguments? #2018-04-14 21:53:37 hagmonk I don't see anyone shoving extra stuff into deps.edn, so I'm somewhat wary about storing build metadata in there #2018-04-14 22:11:17 seancorfield There are pros and cons to both. #2018-04-14 22:12:04 seancorfield If you have a project that expects to be built a particular way, it makes sense to have the build stuff in the project (so an alias for using depstar etc sounds reasonable). #2018-04-14 22:12:28 seancorfield If you have a project that can be used "as-is" but could be built using any tooling available, don't put the build stuff in the project. #2018-04-14 22:12:32 seancorfield That would be my guidance. #2018-04-14 22:14:20 hagmonk thanks @seancorfield … right now I'm patching in clojure.tools.cli support to see how that feels in use #2018-04-14 22:15:40 seancorfield I haven't switched that over to deps.edn yet. It's on my list. #2018-04-14 22:15:55 seancorfield But I also want to switch it to .cljc for ease of maintenance. #2018-04-14 22:16:13 seancorfield (and drop pre-1.7 support) #2018-04-14 22:20:42 hagmonk Hah, I just realized depstar is @ghadi’s handiwork. That's why this code is so tight :) #2018-04-14 23:28:24 hagmonk Ok, it's a bit unpolished, but if folks want to give the thin jar / dependency copying support I added, start here:
clojure -Sdeps '{:deps {depstar {:git/url "" :sha "9524dccdb37ee85b79d3a505447d1e5101f1a73d"}}}' -m hf.depstar.core -h
#2018-04-14 23:34:19 hagmonk The TL;DR version is that -m hf.depstar.core jar works like lein jar, and -m hf.depstar.core deps works like lein-libdir #2018-04-15 00:57:07 alexmiller from my perspective re deps.edn: don’t put arbitrary other gunk in it. #2018-04-15 00:58:55 alexmiller I make no guarantees that arbitrary stuff will get properly merged across multiple deps files and/or transmitted through the code #2018-04-15 01:04:24 hagmonk @alexmiller thanks, that’s a compelling reason to keep things out of there :) #2018-04-15 01:05:28 alexmiller (although I think it probably will right now) #2018-04-15 01:15:11 richiardiandrea @dominicm congrats for the install patch, feels way better. It is also nicer to see the names of the contributors in the log. It was missing before but it goes a long way to "thank" contributors. FWIW, I like this idea. #2018-04-15 02:00:47 alexmiller I have done it a couple times in the past, I just forget sometimes. #2018-04-15 01:15:47 hagmonk Oh cool, TDEPS-26 was fixed too #2018-04-15 02:01:32 alexmiller New releases of tools.deps.alpha and the Clojure tools (clj, clojure) are now available. This is mostly a bug fix release.
To update, use `brew upgrade clojure` on Mac, or the Getting Started installation instructions on Linux.
Windows installers are coming soon(ish)! I have an initial version working and it's about 80% done, so only 80% to go. :)
Updates:
- TDEPS-73 detect invalid deps.edn file and improve error
- TDEPS-61 use Clojars CDN repo
- TDEPS-26 include transitive runtime deps in addition to compile deps
- TDEPS-71 better error if Java not installed - primarily for Linux installer (patch thanks to Mike Fikes)
- TDEPS-65 specify permissions on installed files in Linux installer (patch thanks to Dominic Monroe)
#2018-04-15 02:35:44 seancorfield I ended up not working on clj-new today, but may work on it tomorrow. I still plan to make an announcement about it early next week either way (we have a major infrastructure upgrade on Monday so I suspect it'll be Tuesday when I write up the clj-new stuff). #2018-04-15 02:36:03 seancorfield I did bump the t.d.a. dep to the new release tho' 🙂 #2018-04-15 02:36:21 seancorfield (locally) #2018-04-15 19:47:14 hagmonk so I'm trying to build an uberjar targeting a kafka connector … for the life of me I can't get kafka to load it, it barfs on Could not locate clojure/core__init.class or clojure/core.clj on classpath. #2018-04-15 19:47:35 hagmonk core__init.class is definitely in the uberjar, however #2018-04-15 19:48:31 hagmonk building this as an example with depstar, so I'm trying to figure out if the uberjar put together is somehow wrong #2018-04-15 19:48:44 hagmonk [2018-04-15 19:47:23,641] INFO Loading plugin from: /connectors/postgres-plugin.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:176)
[2018-04-15 19:47:23,646] DEBUG Loading plugin urls: [file:/connectors/postgres-plugin.jar] (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:183)
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.lang.Namespace.<init>(Namespace.java:34)
at clojure.lang.Namespace.findOrCreate(Namespace.java:176)
at clojure.lang.Var.internPrivate(Var.java:153)
at thals.pgconnector.SourceConnector.<clinit>(Unknown Source)
#2018-04-15 19:49:20 hagmonk how's this jar managing to get as far as clojure.lang.Namespace but then claiming it's unable to find clojure/core__init.class … #2018-04-15 19:54:24 hagmonk oh crap, it's the classloader thing #2018-04-15 20:20:01 dominicm Can you link me to the documentation on how kafka loads classes? I'm the author of pack, and I'd like to read about what I'd need to do to support their preferred format. #2018-04-15 20:30:43 hagmonk sure #2018-04-15 20:32:42 hagmonk but, unfortunately, I don't think it's related to the jar format. I believe they start a new classloader and load all the jars into that, which isn't visible to clojure because use-context-classloader is set to true #2018-04-15 20:33:30 hagmonk I'm trying to figure out if there's a way to get around this without resorting to java #2018-04-15 20:35:45 hagmonk the tl;dr version is that they can take uberjars, or a directory full of jar dependencies (excluding kafka runtime goop), including obviously the jar containing the connector implementation #2018-04-15 20:55:11 dominicm The zip version would work better, and fits with the
Pack philosophy. #2018-04-17 02:23:46 cfleming Am I correct in thinking that deps.edn provides no way to build an artifact? #2018-04-17 02:24:29 cfleming Or even to specify co-ordinates for one? #2018-04-17 02:33:46 richiardiandrea I think you understand correctly. Or put in a better way I understand the same. #2018-04-17 02:53:00 dpsutton Have you seen ghadi's depstar? #2018-04-17 02:57:27 cfleming I have, but that just creates a jar #2018-04-17 03:05:42 cfleming Also, is Clojure itself an implicit dependency? The examples don’t add it like lein does. #2018-04-17 03:08:47 dpsutton That does an Uber jar so I think it gets the deps of the clj runner I think #2018-04-17 03:09:07 dpsutton It removes itself though #2018-04-17 03:09:18 dpsutton Depstar itself I mean #2018-04-17 03:09:18 cfleming Oh, I meant in general - if I run a REPL, the Clojure dep comes from the runner? #2018-04-17 03:10:20 dpsutton As far as I can tell yes. I've never seen deps.edn specify a clojure version #2018-04-17 03:11:02 cfleming Interesting. A lot of this is going to be really weird for Cursive support 😐 #2018-04-17 03:12:18 dpsutton It's been rough in CIDER so far as well. It feels a bit like boot. It's not super obvious how to best approach it #2018-04-17 03:12:42 cfleming Yeah, the loose approach to project identity is tricky. #2018-04-17 03:12:45 dpsutton I think @dominicm has done some work towards that end but it's still a little brittle #2018-04-17 03:15:20 dpsutton Kinda like boot you need to know what aliases or tasks to invoke #2018-04-17 03:15:41 cfleming Yeah, boot is tricky for Cursive too 🙂 #2018-04-17 03:17:43 seancorfield @cfleming clj -Sdescribe lists the deps.edn files that are read in. #2018-04-17 03:18:02 seancorfield You can also get that from tools.deps.alpha as clojure-environment (I think). #2018-04-17 03:18:31 seancorfield My bad: clojure.tools.deps.alpha.reader/clojure-env #2018-04-17 03:18:38 cfleming The difficulty is there’s no consistent identifier for a module #2018-04-17 03:18:40 seancorfield (it shells out to clj -Sdescribe tho') #2018-04-17 03:19:36 dpsutton Do you inject your dependencies like cider or do you run another process? I know cursive is big on static analysis #2018-04-17 03:19:53 dpsutton I think we're gonna have to merge edn files from emacs lisp #2018-04-17 05:29:27 dominicm Nope. -Sdeps. I already have this working in viml, elisp will be a doddle #2018-04-17 03:20:05 cfleming @dpsutton I don’t inject any dependencies. #2018-04-17 03:20:18 cfleming Most of Cursive’s functionality is static in the editor. #2018-04-17 03:23:13 seancorfield For clj-new I ended up having to ask clj what deps.edn files it would load, reading them, resolving them, making a classpath, and loading those files... environment (clojure-env)
all-deps (read-deps (:config-files environment))
...
(-> (deps/resolve-deps all-deps resolve-args)
(deps/make-classpath (:paths all-deps) {})
(str/split (re-pattern java.io.File/pathSeparator))
(->> (run! pom/add-classpath))))
#2018-04-17 03:24:41 cfleming Where are git deps cloned to? #2018-04-17 03:25:00 cfleming Is there a local .deps directory or something? #2018-04-17 03:43:05 cfleming Ah, ~/.gitlibs #2018-04-17 03:43:59 cfleming Is it possible to use file:/// URLs with :git/url? #2018-04-17 03:44:08 seancorfield You can use :local/root #2018-04-17 03:44:12 cfleming I can’t get it to work. #2018-04-17 03:44:12 seancorfield (instead) #2018-04-17 03:44:44 cfleming Right, but then I can’t pin a version. #2018-04-17 03:45:03 cfleming (although I’m mostly trying to avoid having to publish to github just to play around 🙂 ) #2018-04-17 03:45:24 seancorfield So :local/root deps just stay where they are on disk, with no version. :git/url is cached to ~/.gitlibs per :sha. :mvn/version is cached to ~/.m2 with a real version. #2018-04-17 03:56:21 alexmiller @cfleming re prior question, the default Clojure version is specified in the install-level deps.edn (which is merged with ~/.clojure/deps.edn and local deps.edn). You can specify it in local deps.edn too if you like. #2018-04-17 03:57:57 cfleming I’m thinking about the case when a Cursive user imports a deps project and maybe doesn’t even have the CLI installed. #2018-04-17 03:58:27 cfleming Where is the install-level stuff put? I guess that’s OS/packager dependent? #2018-04-17 03:59:12 alexmiller yeah, but clj -Sdescribe will tell you that #2018-04-17 03:59:48 cfleming The problem is that IntelliJ, at least on OSX, doesn’t typically know what the users’s shell path is. #2018-04-17 04:00:32 cfleming Will any of this work at all if the user doesn’t have the clj CLI stuff installed? #2018-04-17 04:00:41 alexmiller no #2018-04-17 04:01:17 cfleming That’s a bummer. #2018-04-17 04:01:18 alexmiller I mean, some of it can be used directly via the lib but the lib really works in tandem with the scripts #2018-04-17 04:01:19 cfleming Hmm #2018-04-17 04:02:27 alexmiller the scripts are the bridge from the user’s environment and config files into the lib #2018-04-17 04:02:29 cfleming I’ll have to see if I can detect the install, and consider what to do if it’s not there or I can’t find it. #2018-04-17 04:02:58 alexmiller the scripts are the only layer that know about (for example) classpath caching #2018-04-17 04:03:12 alexmiller but they use the lib to build the cache #2018-04-17 04:03:43 cfleming So there’s no way to programmatically determine the classpath from Clojure/Java? #2018-04-17 04:04:04 cfleming Without exec’ing, I mean? #2018-04-17 04:04:28 cfleming I guess yes, but with no caching. #2018-04-17 04:04:31 cfleming Hmm #2018-04-17 04:04:36 alexmiller there is no simple answer to that #2018-04-17 04:04:58 alexmiller the lib can compute a classpath given a configuration #2018-04-17 04:05:10 alexmiller but the configuration is inherently a product of the environment #2018-04-17 04:05:50 cfleming The environment will be different when scripts are invoked by IntelliJ #2018-04-17 04:06:17 alexmiller to some degree, yes. and I think the question is how much you let the user know that :) #2018-04-17 04:06:46 alexmiller I think the user expectation will be that they get the same effect as running clj in the root of the project #2018-04-17 04:07:28 alexmiller I’ve tried to provide a lot of tools to replicate some or all of what’s in the script layer from inside the lib (Sean is using many of those in the boot tooling) #2018-04-17 04:08:02 cfleming right, but even running clj in the root of the project will be different when run from IntelliJ. #2018-04-17 04:08:45 cfleming Well, the environment will be different, I have no idea if the differences will be significant. #2018-04-17 04:09:57 alexmiller the only things that are critical are current working directory and path #2018-04-17 04:10:18 alexmiller there are potentially some environment variables that can affect things but none of them are likely to be in use #2018-04-17 04:10:20 cfleming path will be the thing that’s different. #2018-04-17 04:10:50 alexmiller can you explicitly add an option to point to the bin dir for clojure? #2018-04-17 04:10:51 cfleming In general OSX doesn’t give apps access to env vars, and doesn’t execute shell init scripts anyway. #2018-04-17 04:11:04 cfleming I think I’ll have to. #2018-04-17 04:11:40 alexmiller on os x, if you install via brew (recommended) clojure will be in /usr/local/bin #2018-04-17 04:12:24 alexmiller well, on linux too if you follow the script #2018-04-17 05:31:27 dominicm Most distro packages won't #2018-04-17 04:12:31 alexmiller windows is … coming #2018-04-17 04:13:34 alexmiller and the install level deps.edn will be at /usr/local/lib/clojure/deps.edn #2018-04-17 04:14:31 alexmiller that last is linux, mac will vary per brew install #2018-04-17 04:15:06 alexmiller mac will be at something like /usr/local/Cellar/clojure/1.9.0.375/deps.edn when installed via brew. maybe there’s a symlink that #2018-04-17 04:15:15 alexmiller ’s version independent, not sure #2018-04-17 04:18:40 cfleming Ok, thanks. #2018-04-17 04:22:28 alexmiller I have tried to do what I can to make things discoverable and computable. if there’s things I can do to help more, I’m open to suggestions #2018-04-17 04:27:43 seancorfield With boot-tools-deps I actually pull the install-level deps.edn from the Clojure brew-install GitHub repo, into the JAR of the library. Looking back, that was probably a bad idea 🙂 #2018-04-17 04:28:13 seancorfield I probably ought to change it to use clojure-env instead! #2018-04-17 04:31:26 alexmiller it’s a bad idea :) #2018-04-17 11:42:35 mfikes @cfleming If it helps, I've been using Cursive for deps.edn projects by first doing clj -Spom and then using the resulting pom.xml #2018-04-17 11:45:31 mfikes I also imagine Cursive might be able to use the output of clojure -Spath, as it gives you the classpath, including paths for any :local/roots or :git/url / :sha deps #2018-04-17 17:03:14 richiardiandrea So...say I have a colleague who does not trust brew install and would like to use the linux script for installing clojure...is it possible to do that? #2018-04-17 17:03:49 richiardiandrea (answer is no, we tried and install -D does not unfortunately work on mac) #2018-04-17 17:13:06 dominicm The install really only does 5 steps, should be easy enough to do them manually. #2018-04-17 17:28:04 richiardiandrea @dominicm yep he probably will do that #2018-04-17 17:40:31 alexmiller you’ll notice the brew install just runs an install.sh in the tar #2018-04-17 17:40:43 alexmiller so you can manually do basically the same thing #2018-04-17 17:42:01 alexmiller with some effort and messiness, install.sh for mac and linux-install.sh could become the same script. I have so far not been able to convince myself that gives me any great value though and there are some useful reasons for them to be different. #2018-04-17 18:08:29 richiardiandrea @alexmiller thanks for answering, was wondering about that as well, good to know #2018-04-17 18:32:05 richiardiandrea so the difference about install on Macosx is only with -d Create directories. Missing parent directories are created as required. No -D (capital). #2018-04-17 18:46:15 dominicm BSD vs GNU install 😔 if we want to work on BSD we should support both really. #2018-04-17 18:53:44 dominicm I've seen some applications probe by using --help and grep #2018-04-17 19:21:55 richiardiandrea I think for mac is even a bit different from BSD, I did not see -D at all #2018-04-17 20:13:15 dominicm Hum, weird, annoying, standards, yay. #2018-04-18 02:39:25 mfikes Planck now joins the deps.edn fray with a new plk script which is literally a rip-off of clj #2018-04-18 02:42:06 alexmiller something something flattery #2018-04-19 00:04:54 zentrope If I have :main-opts ["-e" "(println 'hello)"] I get an EOF while reading error. Is there something I’m missing with the “-e” option? #2018-04-19 00:08:31 gfredericks do you get that with (println :hello) as well? #2018-04-19 00:08:47 zentrope Hm. If there’s any whitespace in the expression at all, the error happens? #2018-04-19 00:09:11 zentrope Yeah, :hello causes the issue, too. #2018-04-19 00:09:50 zentrope However, :main-opts ["-e" "(println,'hello)"] succeeds (note the , for white space). #2018-04-19 00:10:06 gfredericks ah ha #2018-04-19 00:10:29 gfredericks I would call that a bug but I Am Not A Clojure Lawyer #2018-04-19 00:10:43 zentrope It’s certainly principle of greatest surprise. ;) #2018-04-19 00:14:02 zentrope I can’t find it in Jira, if it already exists. Not that I have any skill searching Jira. #2018-04-19 00:30:57 zentrope Yes, it seems major to me, too. #2018-04-19 00:31:54 mfikes Sweet , whitespace hack though. Awesome. #2018-04-19 00:33:14 zentrope If found that in a @seancorfield example for deps.edn somewhere. I wondered why he had that in there. ;) #2018-04-19 00:34:41 seancorfield Yeah, I spent a while bashing my head on the whitespace errors before thinking of , in a drunken moment of defiance, late at night! 🍻 #2018-04-19 00:35:20 gfredericks I have also used , to avoid quoting issues at some point in my life #2018-04-20 03:47:10 mfikes Try as I might, I can't find another way to embed deps EDN in a shebang without the Corfield Comma #2018-04-20 04:40:08 seancorfield I'm honored -- I have punctuation named after me :rolling_on_the_floor_laughing: #2018-04-20 14:23:06 john Corfield comma 😂 #2018-04-20 14:26:30 john @mfikes this is exactly the simple static site generator I was looking for! thanks #2018-04-20 14:27:53 john that's really simple #2018-04-20 14:39:03 dominicm @mfikes just a caution, using the shebang that way won't work on Linux #2018-04-20 14:40:35 mfikes @dominicm Yep. I know. I know. #2018-04-20 14:41:29 mfikes It is a nice pattern for when you want a script to specify its own deps, but alas. #2018-04-20 14:50:14 dominicm oh god #2018-04-20 14:50:18 dominicm I came up with something that works #2018-04-20 14:52:53 dominicm #!/bin/sh
"exec" "clj" "-Sdeps" "{:deps,{markdown-clj,{:mvn/version,\"0.9.85\"}}}" "$0" "
Run as
❯ ./foo.clj '# Hello, world'
I don't know whether to share this or burn it #2018-04-20 14:53:16 dominicm You can kill the commas too, just tested #2018-04-20 14:54:24 mfikes Nice, so the extra stuff there is just interpreted as strings by the compiler that otherwise do no harm 🙂 #2018-04-20 14:54:39 dominicm Bingo #2018-04-20 14:55:18 dominicm I had issues with your md->html, the docs said it was md-to-html? Dunno if in cljs it's different, but thought I would warn you #2018-04-20 15:49:13 timgilbert That is a cool hack (also slightly horrifying) #2018-04-20 15:50:48 timgilbert I'm wondering if various bits of clojure syntax farther down in the file would break bash, but I guess it really does just read line-by-line? #2018-04-20 15:53:59 mfikes My guess is that exec seals the deal, replacing the script with the clj process, thus making it so no further bash interpreting is required #2018-04-20 16:05:59 alexmiller this still pales in comparison to boot’s hack I think - it embeds a jar file in the bash #2018-04-20 16:06:15 mfikes 😱 #2018-04-20 16:06:57 mfikes Like, the binary image? #2018-04-20 16:07:00 alexmiller yes #2018-04-20 16:07:16 alexmiller just go edit your boot script :) #2018-04-20 16:07:25 mfikes Hackers gonna hack #2018-04-20 16:07:33 dominicm fun fact, zip readers scan the file until they find the header. #2018-04-20 16:07:38 dominicm Capsule also has "really executable jars" #2018-04-20 16:07:42 alexmiller yep #2018-04-20 16:19:50 dominicm @mfikes Go ahead! I'm also honoured you took the time to figure out my gh username 🙂 #2018-04-20 16:20:47 dominicm I tested it with a posix-only (ish) shell, so it should work everywhere #2018-04-20 16:24:31 mfikes I forgot about the fact that it needs to be well-formed Clojure code and used single quotes above. Hrm. It still worked. #2018-04-20 16:27:58 dominicm I guess that is well-formed clojure code, you are doing (quote "{:deps …") (quote "$0") 😂 #2018-04-20 16:28:10 dominicm you're not doing what it looks like you're doing, but it works #2018-04-20 16:29:11 mfikes Yep. Wow #2018-04-20 16:29:47 mfikes I think I’ll use \" as it is a little less mystifying #2018-04-20 16:34:21 dominicm It's also way more annoying 😉 #2018-04-20 16:34:22 pesterhazy So much progress in one day! Corfield Comma... Monroe's Shebang #2018-04-20 16:35:50 pesterhazy I think I'd use clojure instead of clj. The latter is wrapped in rlwrap, which seems unnecessary for scripts #2018-04-20 16:51:31 pesterhazy Trying to code-golf: #!/usr/bin/env bash
{ "exec" "clojure" "-Sdeps" "{:deps {markdown-clj {:mvn/version \"0.9.85\"}}}" "$0" " #2018-04-20 16:51:43 pesterhazy not sure that's any better though #2018-04-20 16:53:04 dominicm Isn't it exactly the same? 😂 Although I like the hack #2018-04-20 17:10:08 pesterhazy Yeah, the real insight is in the different interpretation of strings by sh and clojure #2018-04-20 17:24:17 mfikes Oddly, it works with ClojureScript, which I would have expected to take issue with some strings appearing before an ns form
#!/usr/bin/env bash
"exec" "clojure" "-Sdeps" "{:deps {org.clojure/clojurescript {:mvn/version \"1.10.238\"} funcool/tubax {:mvn/version \"0.2.0\"}}}" "-m" "cljs.main" "-re" "node" "$0" " #2018-04-20 18:45:09 ghadi upgraded clj today, from 359 to 375, and I ran into the error Error building classpath. Destination path "my-private-lib" already exists and is not an empty directory
Seems like this popped up recently. #2018-04-20 18:45:44 alexmiller can you explain more about the context? #2018-04-20 18:55:24 alexmiller was this on a linux install or using clj? #2018-04-20 21:24:04 ghadi linux install, using clj #2018-04-21 01:25:03 alexmiller Using prefix dir? #2018-04-23 14:34:33 nicola Hi i have monorepo with submodules - looks like deps does not follow second level of relative paths? Should it? #2018-04-23 14:38:05 nicola cleo/cleo ../backend
fhir/fhir ./sansara/fhir
#2018-04-23 14:38:25 nicola fhir is not found in path #2018-04-23 14:41:23 alexmiller Can you explain your setup? #2018-04-23 14:54:31 nicola ui/
deps.end
backend/
deps.edn
libs/
lib1/deps.edn
lib2/deps.edn
# in ui
backend {:local/root "../backend"}
# in backend
lib1 {:local/root "./libs/lib1"} #2018-04-23 14:54:40 nicola so lib1 is not visible from ui #2018-04-23 14:55:18 nicola And -Spaths is wrong $ui-dir/./backend/libs/lib1/src
$ui-dir/../backend/src
#2018-04-23 14:55:32 nicola looks like it just ignores relative paths #2018-04-23 14:57:31 alexmiller you had “bacend” above - just checking that was a typo here #2018-04-23 14:57:44 alexmiller relative paths should work #2018-04-23 14:58:42 alexmiller and “.libs/lib1” should be “libs/lib1" or “./libs/lib1” #2018-04-23 14:59:37 nicola I’ve tried both 🙂 - will check one more time #2018-04-23 15:01:18 alexmiller clj -Stree might be easier to read too #2018-04-23 15:02:29 nicola Yes i read it #2018-04-23 15:02:37 alexmiller there might be an issue with relative local deps of relative local deps? #2018-04-23 15:02:39 nicola And wrong path is generated 😞 #2018-04-23 15:02:59 nicola it lost .. #2018-04-23 15:03:22 nicola so i’ve got ui/./libs/lib1 #2018-04-23 15:03:41 nicola instead of ui/../backend/./libs/lib1 #2018-04-23 15:05:01 alexmiller I can imagine how that would not work in the code #2018-04-23 15:10:03 nicola Should i switch to absolute paths or try to pull request 🙂? #2018-04-23 15:10:27 alexmiller well you can definitely use absolute paths #2018-04-23 15:10:42 nicola do you have some reader macro to read env in deps.edn? #2018-04-23 15:15:07 nicola Yes, that’s my case - are you planning to support this situation? #2018-04-23 15:20:16 alexmiller off the top of my head, I don’t see why this shouldn’t work #2018-04-23 15:20:22 alexmiller shouldn’t be made to work that is #2018-04-23 15:20:33 alexmiller so yes #2018-04-23 15:20:56 alexmiller I think I know where the problem is #2018-04-23 15:21:04 alexmiller but would be great to have a ticket #2018-04-23 16:29:05 nicola Is there any way to inject env variable into deps.edn? #2018-04-23 16:57:28 delaguardo clj -Sdeps '$(cat deps.edn | echo)' mb? But I never tried that before troll #2018-04-23 17:04:30 dominicm I mentioned this before, it's because the paths are not resolved relative to dependency manifest, but are relative to where the JVM started.
Is it now the case that you're interested in supporting this? 😯 #2018-04-23 17:14:41 alexmiller Sorry, I don’t remember that conversation #2018-04-23 17:28:05 dominicm if tdeps-74 is likely to be fixed, I'm happy. It suits my mono-repo intentions 🙂 ¯\(ツ)/¯ #2018-04-24 09:42:12 pesterhazy Very interesting! #2018-04-24 12:13:25 alexmiller In what way is it official? #2018-04-24 12:48:52 bronsa official by docker #2018-04-24 12:50:06 mfikes Hrm. Seems odd Docker would create an official thing using clojure as its name. (I admittedly don't understand the bigger picture.) #2018-04-24 12:50:26 gfredericks I think it's existed for a while as a leiningen image? #2018-04-24 13:08:48 delaguardo yes, it is. And another for boot as well. And official means that this image is served from the official docker registry and curated by docker maintainers how suppose to take care about upgrading software inside of the image #2018-04-27 12:45:56 niclasnilsson Quick question: Transitive git deps in deps.edn? I.e. a/deps.edn points to b on github and b/deps.edn points to c on github. Doesn’t seem to work without including c in a’s deps.edn. Not implemented or am I doing it wrong? #2018-04-27 12:59:02 hypirion That should work, I've used that myself a couple of times #2018-04-27 13:08:56 alexmiller should work #2018-04-27 13:10:40 alexmiller if you want more information, try clj -Sdeps "{:aliases {:verbose true}}" -R:verbose -Stree, I can probably interpret that a bit more to see what’s up #2018-04-28 01:20:35 cfleming Thanks @mfikes, I’m hoping to add deps support to Cursive soon, that’s helpful. #2018-04-28 01:20:49 mfikes Sweet! #2018-04-28 01:23:09 cfleming Thanks - I haven’t read about any of that yet, but I assumed that Cursive would have trouble with the CLI stuff since IntelliJ doesn’t see shell env vars like PATH. #2018-04-28 01:25:27 mfikes Yeah, in the above, the workaround is to instead launch the IDE from a terminal via open -a "IntelliJ IDEA" #2018-04-28 10:19:43 rickmoynihan Thanks @mfikes I might link to those docs from the README. I’m an Emacs/cider user myself, though am familiar with that problem because of how OSX’s GUI environment doesn’t fork from a shell with your sourced env, like it does under classical Unix systems.
The problems in resolving those file locations were actually the rationale behind letting users provide their own chain of files instead of [:system :home :project], so I’m glad you’ve found that useful; though it’d be good to find a better solution. It’s a shame that tools.deps has to shell out to get that env & I’m definitely open to other solutions; though it’s really more of a problem for tools.deps to solve (if it can). #2018-04-28 13:36:17 mfikes Yeah, I feel like lein-tools-deps is at a point where early adopter community members can actually make use of it and provide feedback related to the various use cases that may arise out there. #2018-04-28 17:23:05 potetm fwiw I’ve gotten by thus far by generating a pom (via clj -Spom) — you don’t get auto-importing, but it’s a very clear execution model #2018-04-28 17:25:29 mfikes Yeah, I think one new thing that lein-tools-deps gets you is Git deps (`clj -Spom` doesn't seem to support that) #2018-04-28 17:26:36 potetm ah yeh, makes perfect sense #2018-04-30 01:19:38 hagmonk the problem is that lein takes the top level list of :exclusions in project.clj and applies those to all dependencies, and seemingly to all transitive dependencies #2018-04-30 01:20:52 hagmonk it doesn't appear that deps.edn has a facility to achieve the same thing, meaning that one has to manually review the dependency graph, and if any transient dependencies include an unwanted dependency, one must include that dependency and put a specific exclusion in the maven coordinate #2018-04-30 01:34:29 alexmiller That’s true #2018-04-30 01:38:31 hagmonk Is there a more "tools.deps-ish" way to solve this problem? #2018-04-30 01:56:06 alexmiller No #2018-04-30 02:00:16 alexmiller This is not a common issue so I don’t see it very high on the priority list #2018-04-30 02:01:12 hagmonk Hmm, it's an issue I often run into when trying to "rationalize" the logging situation in a project that has lots of Java deps #2018-04-30 02:01:51 alexmiller One thing that may exist at some point is the means to specify a lib removal. Given that, this could be done with override-deps or possibly a root dep exclusion #2018-04-30 02:04:07 hagmonk It's kind of gross that the presence of certain deps on the classpath causes side effects for logging frameworks. Perhaps this is best handled by a small tool that rewrites deps.edn to incorporate the necessary inclusions / exclusions #2018-04-30 13:14:41 pesterhazy @hagmonk wow log.dev looks great, I never knew that existed #2018-04-30 16:32:27 hagmonk @pesterhazy yeah! since I end up importing a lot of Java deps I frequently start projects with this set of inclusions / exclusions just so I know that Java logging will be sane #2018-05-03 14:35:58 mfikes ^ demo of using some latest WIP for lein-tools-deps within Cursive #2018-05-03 14:37:00 alexmiller what’s that’s :system :home :project from? #2018-05-03 14:37:21 alexmiller because I would call those :install :user :project instead #2018-05-03 14:38:06 alexmiller is that lein-tools-deps? #2018-05-03 14:38:56 alexmiller cc @rickmoynihan ^^ #2018-05-03 14:43:23 dominicm @alexmiller is lein-tools-deps relying on implementation behaviour in order to label those things, based on positional information? #2018-05-03 14:43:48 alexmiller presumably #2018-05-03 15:09:43 rickmoynihan @alexmiller: Yes, it’s lein tools deps. #2018-05-03 15:10:28 rickmoynihan Yes, good idea those are better names I’ll create a ticket to rename them. #2018-05-03 15:11:37 rickmoynihan @dominicm: It does indeed. Do you know of a better way right now? #2018-05-03 15:11:52 dominicm No, there's a ticket open on JIRA. I'm waiting on it #2018-05-03 15:12:09 rickmoynihan do you have the ticket number? I’ll create an issue to update lein-tools-deps when tools deps improve it. #2018-05-03 15:12:43 rickmoynihan ta #2018-05-03 15:14:18 dominicm In my case, I want to allow users of pack to run pack from outside their main deps.edn if they like, which means replacing the project-deps. #2018-05-03 15:16:20 alexmiller we’re still kicking around this stuff #2018-05-03 15:17:11 alexmiller from one perspective there are fixed things and from another tools.deps doesn’t ultimately care where they come from. Right now the scripts know about specific locations, but tools.deps doesn’t. #2018-05-03 15:18:09 alexmiller but there are some places where it would be useful to know the context, for example when resolving deps.edn files in local transitive deps, you want to interpret that in the proper context (with install and user but without the initial dep project deps.edn) #2018-05-03 15:18:35 alexmiller and probably without -Sdeps #2018-05-03 15:20:51 rickmoynihan @alexmiller yeah I agree entirely. #2018-05-03 15:21:10 alexmiller I am taking the long hammock on the best resolution to this #2018-05-03 15:21:33 rickmoynihan haha very wise 🙂 #2018-05-03 15:21:35 alexmiller we’ve got some datomic tooling that wraps around all this and has some of the same tensions #2018-05-03 15:21:53 alexmiller build tooling that is #2018-05-03 15:23:11 rickmoynihan I’ve been meaning to ask on macos with the brew installer, it looks like /usr/local/bin/clj etc are sym linked to the latest version; would it not be possible for /usr/local/etc/deps.edn to also symlink to the latest system deps.edn?
It looked link the linux installer puts system in a well known location. #2018-05-03 15:23:37 dominicm that's a bit exciting, datomic + tools.deps seemed unlkkely to me as I expected java to be involved. #2018-05-03 15:24:46 alexmiller on macos, some of that is dependent on how you use brew I think. while it will usually be there, you can change your prefix dir for brew #2018-05-03 15:24:51 rickmoynihan @mfikes: sorry I haven’t got around to looking at your PRs yet btw… I’ll try and get around to it. #2018-05-03 15:24:52 dominicm @rickmoynihan don't depend on where the linux installer puts it, some distro packaging requirements put it elsewhere. #2018-05-03 15:24:54 alexmiller and similarly we now accept a prefix dir for linux #2018-05-03 15:25:08 alexmiller @dominicm this is related to how datomic cloud is built #2018-05-03 15:25:27 alexmiller wrt dependency fetching #2018-05-03 15:26:00 dominicm NixOS and GoboLinux certainly don't use /usr at all. #2018-05-03 15:26:21 alexmiller eventually some of that whole system may percolate into something public and generally useful but not sure #2018-05-03 15:26:25 rickmoynihan @dominicm: Yeah I know… just thinking an acceptable approach for lein tools deps, might be to do some default checks (in a fallback chain) with no explicit config - but let users override. #2018-05-03 15:27:29 dominicm @rickmoynihan I would expect most linux users to not be aware of where their distro chooses to put these files. #2018-05-03 15:27:32 alexmiller certainly, the install-specific location is a pain #2018-05-03 15:27:34 rickmoynihan would obviously prefer not to do that #2018-05-03 15:28:37 alexmiller one option would be to get rid of the install file and move it into the code #2018-05-03 15:28:42 rickmoynihan I’d expect most linux users would; or would be able to find out. They can always use clj -Sdescribe, which can be documented in an FAQ for lein-tools-deps #2018-05-03 15:29:09 alexmiller if the base is loaded as a resource, then the scripts no longer care #2018-05-03 15:29:58 rickmoynihan that’s a great idea #2018-05-03 15:30:10 alexmiller likewise, you could restrict ~/.clojure/deps.edn to aliases only #2018-05-03 15:30:28 rickmoynihan resolving user is a lot easier too so install is the main problem #2018-05-03 15:30:56 alexmiller well, not only aliases, but not :deps/:paths #2018-05-03 15:31:44 rickmoynihan @alexmiller: sorry not sure why that’s an issue… #2018-05-03 15:32:11 alexmiller it goes to what you need to consider when reading transitive deps.edn, and also to what -Srepro means #2018-05-03 15:32:29 alexmiller there are a bunch of these things that interrelate #2018-05-03 15:33:12 rickmoynihan ok I see what you mean #2018-05-03 15:43:38 dominicm That would solve some issues I have in pack with reproducibility, I don't look at the system deps.edn at all. #2018-05-03 16:23:28 seancorfield boot-tools-deps includes a version of the system deps.edn that gets pulled down from GitHub and baked in at build time. While that means you can technically use boot-tools-deps without needing clj etc installed, it's also a very bad idea: if you have clj installed, it could have a different system deps file and you could get different behavior. #2018-05-03 16:24:04 dominicm @alexmiller I am a little confused by what you were saying about building datomic cloud, are you able to elaborate. #2018-05-03 16:28:58 alexmiller no #2018-05-03 16:29:24 alexmiller it’s all internal and I didn’t work on it #2018-05-03 16:24:24 seancorfield Given that I often tell folks boot-tools-deps is only intended to add Boot-like functionality to an existing clj project, I should start relying on clj -Sdescribe (via the API call in tools.deps.alpha). #2018-05-03 16:26:47 mfikes One problem with the API call is that clojure needs to be in your PATH. #2018-05-03 16:27:27 mfikes So, this causes an issue if you happen to run things from Cursive on macOS, which ends up with a degenerate path that doesn't include /usr/local/bin #2018-05-03 16:30:17 alexmiller I think given the changes I described above you may be able to avoid all of those #2018-05-03 16:30:39 alexmiller in that tools.deps.alpha would already include the necessary stuff #2018-05-03 16:30:55 mfikes (In case it is useful, here is the degenerate path that Cursive gets: /usr/bin:/bin:/usr/sbin:/sbin) #2018-05-03 16:31:15 mfikes Yeah, TL;DR is stuff is baked in, no need to really run clojure, etc. #2018-05-03 16:31:22 alexmiller the tradeoff is that the install deps.edn then becomes “magical” in that it’s hard to see what’s in it #2018-05-03 16:32:10 mfikes Definitely calls for some hammock time #2018-05-03 16:32:20 dominicm I imagine in enterprisey environments you would want to have a system override of maven/repos #2018-05-03 16:32:34 alexmiller 💤 #2018-05-03 16:34:51 seancorfield Changing the subject slightly... at work we have a mono-repo with multiple subprojects and we're starting to convert those over to deps.edn from Boot (with external dependencies in EDN files, using a different/simpler format). One of the problems we have to tackle is that, currently, we use a version.properties file to "pin" version numbers for libraries (each line is of the form: group/artifact=x.y.z to specify the version). #2018-05-03 16:35:58 seancorfield Since clj requires that :override-deps be in an alias (yes?), I created a versions/deps.edn containing a :defaults alias with :override-deps for everything we need, but then it's a little clunky to run: clj -Sdeps "`cat ../versions/deps.edn`" -A:defaults:test:runner
#2018-05-03 16:36:21 seancorfield If clj allowed us to specify extra config files, that would be nicer. Thoughts? #2018-05-03 16:40:52 dominicm Alias only files would solve that issue too I guess, if they were searched for in parent directories. #2018-05-03 17:14:44 alexmiller so the :default-deps is really about this problem of dep mgmt in the large #2018-05-03 17:14:58 alexmiller but it is not a fully formed (or working) idea #2018-05-03 17:15:49 alexmiller so I guess I’m saying - we see this problem and think it’s important. #2018-05-03 17:16:13 alexmiller but there’s not a complete answer there yet. #2018-05-03 17:24:54 seancorfield Good to know. Thanks @alexmiller! #2018-05-03 17:25:30 seancorfield Our transition from Boot to clj / deps.edn is likely to be very slow/gradual because of stumbling blocks like this. #2018-05-03 17:28:04 seancorfield Our monorepo has about 30 subprojects with all sorts of cross-dependencies. Right now we have a Boot task that uses tools.namespace to deduce dependencies based on namespace usage, so we don't have to explicitly manage those cross-subproject dependencies, but as we're restructuring code, I'd like to see that become more explicit and streamlined. #2018-05-03 17:29:19 seancorfield (we built the namespace-based dependency analyzer because we initially had some strict rules about how code in a few of those subprojects could use code in other subprojects, to maintain separation via APIs etc -- but we've changed our position on that now) #2018-05-03 17:31:50 dominicm I had been excited to see what you would invent for managing the deps.edn files using the automatic dependency analysis. I'm glad to hear someone else is going down the same path as us. #2018-05-03 17:36:36 seancorfield Yeah, we've talked about using the existing analyzer code to generate the cross-dependencies as deps.edn files but I think I want those human-editable with comments etc and I don't want to deal with any Clojure source reader complex enough to retain whitespace/comments 🙂 #2018-05-03 17:39:04 dominicm If you did want to go down the whitespace/comment retention, I had surprisingly good success writing the injector for pack with rewrite-clj: https://github.com/juxt/pack.alpha/blob/master/src/mach/pack/alpha/inject.clj 140 lines gets an entirely whitespace-safe modification of the deps.edn. It even does the funky vertical alignment of parameters if you do that in the same map at any point. #2018-05-03 18:06:27 seancorfield Yeah, I looked at that code -- and that convinced me I don't want to do that 🙂 #2018-05-03 21:26:23 rickmoynihan @seancorfield / @dominicm can you describe those dependency analysis systems at all? What exactly are they doing, and what problems are they solving? Are you building multiple applications? If so is a limitation not that all applications in the same mono-repo need to use the same version of any given namespace?
If that’s the case what is the motivation for doing the analysis at all? i.e. why not just package everything into the same jar, and rely on the entry point determining the loaded namespaces? Is it because some of your libraries/apps are opensource and others aren’t?
Curious because we’re planning to move towards more of a mono-repo approach too. #2018-05-03 21:32:11 seancorfield As an example, we have subprojects for low-level systems-y stuff, data mapping (our wrappers for JDBC), a core domain subproject, a web utilities subproject, and so on. Then we have subprojects for each actual app (api, auth, login, billing, etc). So any given app subproject relies on a particular combination of other subprojects (most of them) but we also have background processes that don't need any of the web stuff etc. The tools.namespace analysis looks at the project you want to build, finds all the namespaces in it (across all subprojects) and the list of namespaces they all require, transitively (the basic graph analysis, treating everything as "one big app"), then it takes the project you want to build and starts from those namespaces to identify which subprojects contain namespaces that are (transitively) required, then the dependencies EDN file in each of those subprojects are read to generate the full list of external libraries needed. #2018-05-03 21:33:14 seancorfield Does that explain it well enough? #2018-05-03 21:35:46 seancorfield The two main goals were so that we didn't have to explicitly write down the subproject dependencies (they change automatically as code requires a different set of namespaces in other subprojects) and we wanted to enforce certain code path exclusions (e.g., no apps are supposed to depend on billing directly, they're only supposed to use the billing-sdk, billing apps shouldn't use the code domain library except via a specific "DMZ"-style namespace that defines the API). #2018-05-03 21:37:00 seancorfield That second goal is no longer important as we've substantially changed our deployment / infrastructure plans since mid-2017. And, without that, the benefit of auto-tracking dependencies is questionable -- I'm leaning toward explicit dependencies more now. #2018-05-03 21:40:45 rickmoynihan ok so every app uses the same version of everything in the monorepo? #2018-05-03 21:41:47 rickmoynihan Why not to just put all the src paths into one big classpath, uberjar that, and rely on the require-tree to just shake out what’s not used?
Is it because you also wanted the rules to be enforced? #2018-05-03 21:42:12 seancorfield Yes, the rules were important when we started down this path. #2018-05-03 21:42:56 seancorfield We can't quite make an uberjar of the whole thing because there are some subprojects that do rely on different versions of external libraries. #2018-05-03 21:43:35 seancorfield But we could uberjar most of it into one big JAR and start it with -m ws.whatever.namespace. #2018-05-03 21:43:39 rickmoynihan sub libraries, or sub apps? #2018-05-03 21:44:36 seancorfield Our core domain subproject has several -main entry points already for background processes and cron tasks. #2018-05-03 21:46:08 seancorfield The auto-tracking of dependencies makes it a lot easier to move code into new subprojects (keeping the ns the same) which we've done quite a bit as we've teased apart some of the legacy code from the newer code. But overall it wouldn't be much work to just manually update deps.edn in each subproject as/when we do that. #2018-05-03 21:47:25 rickmoynihan and each of those -main entry points has different external library versions?
Sorry trying to understand whether you consider multiple -main’s the same app, or is it what makes a different app that it has the same library versions / classpath, or something else? #2018-05-03 21:48:07 rickmoynihan i.e we have some apps with multiple -main entry points… but they’re really different aspects/modes of the same app.
We also have different apps altogether #2018-05-03 21:49:42 seancorfield For the most part, each web app is its own separate subproject, since we deploy different apps on different servers. A lot of the background stuff runs a different server (well, two servers), so we want it separate from the web apps, but mostly all together for the server deployment. #2018-05-03 21:50:54 seancorfield We have some different dependencies in our web apps: one is heavy on websockets, one uses Bidi, the rest don't. We have two web apps that use netty/socket-io but have to use different versions because ... history ... #2018-05-03 21:51:59 seancorfield We also have a couple of subprojects that are only used from our legacy code base, so we want to keep that code physically separate so we can just delete it once we retire the legacy apps. #2018-05-03 21:52:21 rickmoynihan yeah, that’s an issue for us doing what I was suggesting… i.e. the transitive dependencies start to conflict and then you’re in a world of pain #2018-05-03 21:53:50 seancorfield Right, we're in the situation where the dependencies would only conflict in the top-of-tree subprojects, not the lower-level stuff. #2018-05-03 21:54:34 seancorfield (and that's because we've historically "pinned" the versions of most of our external libraries) #2018-05-03 21:46:08 dominicm We can't do as you propose due to different deployment environments meaning that certain code paths completely don't work.
It's also nice to be able to actually remove one of the two hundred dependencies every now and then, which is easier when they're limited to certain boundaries.
Our deployable files are massive now. #2018-05-03 21:46:44 seancorfield Our JAR files are about 30-40MB each. #2018-05-03 21:47:37 dominicm There's a lot of substituting in our projects. So running A might have it's iProtocol that's incompatible with B. #2018-05-03 21:48:01 dominicm Our files are 5x that #2018-05-03 21:53:28 rickmoynihan @dominicm: can you explain what you mean by deployment environments influencing the code paths working? #2018-05-03 21:59:51 rickmoynihan I’m hoping to manage our apps as independent projects in the same mono repo; with some shared support libraries alongside; where all apps will effectively share the same version of the supporting code… but apps may have different 3rd party deps. #2018-05-03 22:04:16 rickmoynihan Thinking out loud; I’ve been wondering how this stuff fits with something like integrant… though it could well be a terrible idea #2018-05-03 22:10:09 rickmoynihan actually it is a terrible idea #2018-05-04 05:30:12 dominicm Integrant with aero is basically how we solve some of this. We built something that looks like Integrant using component. #2018-05-04 08:13:09 rickmoynihan Yeah, we use integrant too; in one project via duct (so weavejester can write all the middleware config layer for us) and in another older project we retro-fitted integrant with aero… which in many ways I prefer to duct; but there are pros and cons.
What I was really getting at though with the integrant thought was that you could possibly imagine combining integrant and deps.edn into a hybrid that would give you dependency resolution on components and configure/wire them together at the same time.
I find it interesting how tools like integrant/component essentially make your require tree composable and configurable, and wonder if there’s a way to use integrant for packaging too. i.e. to turn integrant components (modules) into individually resolvable artifacts that something like tools.deps could then fetch and resolve for you.
It’s currently a very half baked notion, and most likely a terrible idea though 🙂 #2018-05-04 05:33:18 dominicm Our problem is pretty much that a namespace can blow up if deployed where it can't read certain environment variables and look at certain files. These files change per library, with the same path. I guess there's some convention for the path. This is particularly noticeable in localization. #2018-05-04 07:23:52 sveri @rickmoynihan In our team using gradle we define every dependency explicitly, no matter if its an in-team-project dependency or an external one. So naturally I would try the same with leiningen / tools-deps / boot. #2018-05-04 08:17:41 rickmoynihan @sveri: Not sure exactly what you mean by defining every dependency explicitly? Do you just mean that you produce mvn artifacts for each subproject? #2018-05-04 09:15:14 sveri @rickmoynihan gradle has a concept of project dependencies. In the root build file you can define your projects which then can depend on another. #2018-05-04 14:51:59 alexmiller this is kind of a wip / sneak-peek but I’ve implemented a dynamic add-lib that you can use at the repl (that also will download and add transitive deps) #2018-05-04 14:52:12 alexmiller $ clj -Sdeps "{:deps {org.clojure/tools.deps.alpha {:git/url \"" :sha \"d492e97259c013ba401c5238842cd3445839d020\"}}}"
Cloning:
Checking out: at d492e97259c013ba401c5238842cd3445839d020
Clojure 1.9.0
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'org.clojure/core.memoize {:mvn/version "0.7.1"})
Downloading: org/clojure/core.memoize/0.7.1/core.memoize-0.7.1.pom from
Downloading: org/clojure/core.cache/0.7.1/core.cache-0.7.1.pom from
Downloading: org/clojure/data.priority-map/0.0.7/data.priority-map-0.0.7.pom from
Downloading: org/clojure/core.memoize/0.7.1/core.memoize-0.7.1.jar from
Downloading: org/clojure/core.cache/0.7.1/core.cache-0.7.1.jar from
Downloading: org/clojure/data.priority-map/0.0.7/data.priority-map-0.0.7.jar from
true
user=> (require 'clojure.core.memoize)
nil
user=> (add-lib 'clj-time {:mvn/version "0.14.3"})
true
user=> (add-lib 'clj-time {:mvn/version "0.14.3"})
false #2018-05-04 14:52:32 alexmiller ^^ you can use that command at the top to try it yourself off the branch #2018-05-04 14:53:18 alexmiller everything about this is subject to change as Rich and I are still discussing many aspects of naming and use #2018-05-04 14:56:17 alexmiller there is also a new libmap namespace that tracks the libs currently in use (these can be loaded via a property that points to a file which is set by clj, or by a property set to edn data - hook for tools, or by just inspecting the classpath and trying to understand it) #2018-05-04 14:57:35 alexmiller add-lib will add new libs in the context of the existing libs, so it won’t (for example) download and add libs of transitive deps that have already been loaded. when you add libs, those are added to the runtime lib map and considered in subsequent adds #2018-05-04 14:59:09 dominicm This is very cool, and useful! #2018-05-04 14:59:31 alexmiller the primary use here is to add a capability to the repl environment to dynamically add libs. this does not give you the isolation of something like pods, but it’s a small useful step into a more live repl experience. #2018-05-04 15:00:39 dominicm @alexmiller this uses java properties you say? #2018-05-04 15:01:05 alexmiller yes - clojure.libfile (will now be set by clj) and clojure.libmap #2018-05-04 15:02:40 alexmiller although the fallback to classpath introspection actually does a reasonable job even if that’s not set in my experience so far #2018-05-04 15:03:24 dominicm Pods may conflict with that, as java properties aren't localized to the pod. I don't have an alternative suggestion, but just a data point that occurs to me. #2018-05-04 15:04:34 alexmiller if you’re using pods, you don’t need this #2018-05-04 15:05:02 alexmiller classpath introspection: #2018-05-04 15:05:02 dominicm I'm thinking of inspection of clojure.libfile for other purposes, e.g. finding gitlibs to ignore during a (reset) #2018-05-04 15:05:49 dominicm That's a bad example in the context of pods, but I think it conveys my intent. #2018-05-04 15:06:42 alexmiller the clojure.libfile points to the libfile corresponding to the classpath used when starting the jvm. that has a fixed answer. #2018-05-04 15:07:08 alexmiller if you’re using pods, you’re in a bubble under that. #2018-05-04 15:07:28 dominicm libmap then, rather than libfile? #2018-05-04 15:08:04 alexmiller libmap is still jvm-wide so like libfile #2018-05-04 15:08:28 alexmiller but I think it’s interesting to consider how to allow classloaders to build their own env #2018-05-04 15:08:45 dominicm Maybe I don't understand what libmap is, what is it? #2018-05-04 15:08:57 alexmiller it’s just libfile in edn form #2018-05-04 15:09:27 alexmiller so a tool could start the jvm, build the classpath via means other than clj and pass the libmap via data rather than file #2018-05-04 15:10:43 dominicm Given the definition of "jvm-wide", then java properties are always right. My thoughts are that it's really closer to being about the classloader though, what libmap generated the current classloader. #2018-05-04 15:11:44 alexmiller the runtime libmap is basically stored in a var right now. maybe interesting to consider whether that should be dynamic to allow per-thread context or something per-classloader to handle pod/clj-embed type cases #2018-05-04 15:12:05 dominicm Sorry, this probably sounds very pedantic. This is really exciting, I'm happy about the feature even as it currently is. #2018-05-04 15:12:43 dominicm It's definitely notable that due to how Clojure works, embedding always carries a new copy of Clojure. So "clojure local" is perhaps sufficient? #2018-05-04 15:33:31 alexmiller no, I appreciate the good pokes on this. we’ve thought a fair bit about the add-lib stuff but less about the runtime management of the libmap and so I’m still trying to work out the ideal shape of that. and we have other deeper plans for the libmap stuff so I want to get it right. #2018-05-04 15:33:51 alexmiller it’s reasonably likely that part will actually float into Clojure itself #2018-05-04 15:34:39 dominicm Which part, sorry? #2018-05-04 15:35:44 alexmiller the runtime libmap management (not add-lib) #2018-05-04 15:37:23 dominicm So clojure.core will be aware of the deps.edn format? 🤔 #2018-05-04 15:37:42 alexmiller it’s just a map #2018-05-04 15:37:51 alexmiller so, yes it will be aware of maps :) #2018-05-04 15:39:31 alexmiller in terms of doing semantic things - that doesn’t have to be in core. core’s job is to ensure that it’s loaded and available in a way that can be replied upon #2018-05-04 15:39:52 alexmiller I wouldn’t rule out core being aware of it but I don’t see a need for that yet #2018-05-04 15:40:22 alexmiller well, maybe. we’ll see where things go. :) #2018-05-04 15:42:06 alexmiller maybe I should push this stuff into a blog post #2018-05-04 15:47:23 dominicm I'm a little confused. Will it be a state storage, which libmap will sit in as a key? #2018-05-04 15:48:01 dominicm Blog posts are always exciting, even if marked as "these ideas are alpha" #2018-05-04 15:48:42 alexmiller I don’t think I understand the question #2018-05-04 15:48:54 alexmiller right now it’s a var holding an atom holding the libmap #2018-05-04 15:49:25 alexmiller you can look at the code on the branch, it’s not that long #2018-05-04 15:50:24 dominicm What will Clojure know about managing the runtime libmap though? I'm really confused by what it means for that to be in Clojure, sorry. #2018-05-04 16:29:53 alexmiller it means that anything using Clojure could look at the libmap (not just things using tools.deps.alpha) #2018-05-04 16:42:25 dominicm I see! So there would be a var which was mutated or something? #2018-05-04 16:37:15 seancorfield The channel name is #tools-deps not #tools.deps BTW @alexmiller (last line of the blog post) #2018-05-04 16:37:35 alexmiller gosh dang it #2018-05-04 16:38:38 alexmiller fixed #2018-05-04 16:56:35 alexmiller it’s a var holding an atom, so would just swap! the atom #2018-05-04 16:57:20 alexmiller but I think there are interesting potential uses that are merely read-only #2018-05-04 17:09:33 dominicm I understand now. I was interpreting your words too broadly I think. This would solve the problem of pods.
If I'm thinking right, this would complicate Clojure becoming aware of class loaders in the future. Admittedly, this is something I am a minority who longs for this. #2018-05-04 17:25:10 alexmiller Clojure only works because of custom classloaders so I don’t think I agree at all with that #2018-05-04 17:25:29 alexmiller I also don’t think this would solve the isolation aspect of pods #2018-05-04 19:43:45 dominicm I mean that you can't require multiple versions of a namespace from different class loaders.
No? The pod could be controlled to update the atom on start couldn't it? #2018-05-04 20:02:38 alexmiller I think you’re misunderstanding what is happening here #2018-05-04 20:02:52 alexmiller but I’m not sure that I understand what you think it’s doing enough to clarify #2018-05-04 20:03:14 alexmiller the libmap is not affecting anything about how the Clojure runtime currently works #2018-05-04 20:05:57 alexmiller it is assisting in opening a path to additively add jars to the DynamicClassLoader in use (this actually has existed in Clojure for a very long time with clojure.core/add-classpath which is deprecated and I think has not worked for a very long time) #2018-05-04 20:08:14 alexmiller pods in Boot provide isolation (different Clojure runtimes) and already do allow you to load different versions of a Clojure namespace at the same time in different pods as far as I understand them #2018-05-04 20:12:04 dominicm My use of from there was poor. You can't have multiple versions of a library loaded into a single Clojure. You have to load a whole new Clojure. #2018-05-04 20:14:07 alexmiller yes and nothing is changing about that #2018-05-04 20:14:43 alexmiller I’d say that problem is way down the interest list #2018-05-04 20:16:14 dominicm I know. I was thinking about it that were ever to be "fixed". Yeah, I'm one of the few it bothers because I have interest in a constrained deployment environment, where dependencies aren't really possible. #2018-05-04 20:22:12 alexmiller I totally get why and when isolation is useful, but Clojure was never designed with that intent and there seem to be more important problems to solve #2018-05-05 08:25:04 slipset Reason I’m asking is sort of at least twofold:
1) If it’s sort of the same, is there anything to be learned by the clj-refactor impl
2) If it’s sort of the same, can add-lib be used as a replacement for the clj-refactor impel so we get consistent behavior.
3) most likely just an off by one error #2018-05-05 08:26:32 dominicm I'm certain it's almost entirely the same. I expect that in a few months this implementation will find it's way into refactor-nrepl. #2018-05-05 19:11:28 hagmonk @alexmiller Riddle me this: how does one add a maven dep and the corresponding sources and javadoc jars to the classpath? (needed for cider javadoc to work) #2018-05-05 19:13:15 alexmiller Not supported #2018-05-05 19:13:58 alexmiller Currently you can only have one artifact per library #2018-05-05 19:14:39 alexmiller I’m not opposed to supporting it, just trying to figure out how #2018-05-05 19:15:34 hagmonk Got it #2018-05-05 19:25:00 dominicm @alexmiller would treating it as a unique library be insufficient? #2018-05-05 19:28:33 hagmonk I'd propose clojure.tools.deps.alpha/merge-alias-rules be extended with some way to delegate merge strategy to the extension type #2018-05-05 19:29:47 hagmonk Uh, maybe not, that is only used to merge the same map key between different deps.edn sources #2018-05-05 19:31:13 hagmonk maybe clojure.tools.deps.alpha/resolve-deps instead - when merging :extra-deps into :deps #2018-05-05 19:34:05 hagmonk you still end up spitting out a map though, perhaps the key of that map needs to change from being the dependency path, to an opaque composite key that represents "a unique artifact". The dep path itself could move into the value (during resolution, not in deps.edn itself) #2018-05-05 19:36:42 alexmiller That’s only used for alias types, which are not extensible #2018-05-05 19:37:23 alexmiller The merge-alias-rules that is #2018-05-05 19:38:08 alexmiller Treating it as a unique library isn’t possible as the group/artifact is used for resolution #2018-05-05 19:38:50 alexmiller This needs more thought and I consider it a lower priority than other stuff on the plate #2018-05-06 19:32:29 hagmonk Ok, another riddle for you guys #2018-05-06 19:33:54 hagmonk If I include :mvn/repos in deps.edn for project A, because one of the coordinates has to come from a private maven server, then I depend on project A from project B via :local/root, clojure -Sdeps dies saying it can't find the artifact in the private repo #2018-05-06 19:35:53 hagmonk I keep running into these instances that are encouraging me to go back to producing a maven artifact, rather than using :local/root or :git/url #2018-05-06 21:36:07 alexmiller This is a known bug, haven’t had a chance to fix it yet #2018-05-06 22:20:31 hagmonk I created a little tool to help me convert troublesome project.clj files into deps.edn. There are probably a dozen of these but I couldn't find one that actually worked for me :) #2018-05-06 23:07:31 mfikes @hagmonk That's cool. I tried seeing what would happen if you made it run directly as a git dep, but the command line gets quite large https://github.com/mfikes/depify#usage #2018-05-06 23:10:40 hagmonk @mfikes yah, I thought about doing that too, and wondered if you could specify the git coordinate + -Sresolve on the command line and make it work ... haven't tried yet #2018-05-06 23:10:57 hagmonk git coordinate as in the repo location + ref #2018-05-06 23:24:57 hagmonk ah: -Sresolve-tags, and no that doesn't work, since it wants to mutate the deps.edn file ¯\(ツ)/¯ #2018-05-08 18:24:11 richiardiandrea I have a question, with this deps.edn:
{:paths ["src"]
:aliases {:dev {:extra-paths ["test"]}
:self-host {:extra-deps
{andare {:mvn/version "0.9.0" :exclusions [org.clojure/clojure]}
mount {:mvn/version "0.1.13-SNAPSHOT"}}}}}
and command:
clojure -Srepro -A:dev:self-host -Stree
shouldn't be org.clojure completly excluded from the tree? #2018-05-08 18:24:28 richiardiandrea (`mount` does not include it) #2018-05-08 18:25:01 alexmiller -Srepro is kind of in an in between state right now - it’s actually including the install deps.edn, but not ~/.clojure/deps.edn. #2018-05-08 18:25:12 alexmiller and the install one includes clojure #2018-05-08 18:25:14 richiardiandrea uhm... #2018-05-08 18:25:22 alexmiller I do not expect this to always be the case #2018-05-08 18:25:24 richiardiandrea ok that answers it #2018-05-08 18:25:36 alexmiller but the meaning and use of -Srepro is kind of a topic of internal debate atm #2018-05-08 18:26:02 richiardiandrea so at the moment is there a way to work in isolation? without any interference with the global env? #2018-05-08 18:26:36 alexmiller there is no way to completely exclude the install-level clojure dep #2018-05-08 18:26:55 alexmiller (although clojure doesn’t run without clojure so I didn’t expect that to be an issue) #2018-05-08 18:28:05 alexmiller what are you trying to do? #2018-05-08 18:29:06 richiardiandrea I am working with self-host ClojureScript so I don't need Clojure for running, but having it on the classpath might cause issues #2018-05-08 18:29:22 richiardiandrea let me debug and see if that's the issue though #2018-05-08 18:29:38 richiardiandrea but in general you probably might want to consider the use case #2018-05-08 18:52:22 alexmiller clj runs clojure.main (always) which requires a class on the runtime. How does self hosted avoid that? #2018-05-08 18:53:03 alexmiller That is, the entire purpose of clj is to run a class on the JVM #2018-05-08 18:54:20 dominicm @alexmiller I'm guess -Spath is being used only #2018-05-08 20:36:09 richiardiandrea Yes I am using it only for building the classpath #2018-05-08 20:36:31 richiardiandrea And I am using clojure for it #2018-05-08 20:42:57 alexmiller well, you could avoid this by invoking tools.deps directly too #2018-05-08 20:43:27 alexmiller that is, as a library, rather than via the script #2018-05-08 20:45:30 alexmiller if you use -A:deps that will add tools.deps.alpha and its dependencies to your classpath. you could even invoke the clojure.tools.deps.alpha.script.make-classpath script directly #2018-05-08 20:54:44 alexmiller clj -A:deps -m clojure.tools.deps.alpha.script.make-classpath --config-files /path/to/whatever.edn --libs-file /path/to/out.lib --cp-file /path/to/out.cp --jvm-file /path/to/out.jvm --main-file /path/to/out.main -R :self-host #2018-05-08 20:55:12 alexmiller something like that - I think that script requires all of those and that they are absolute paths #2018-05-08 21:06:59 richiardiandrea Ok that is a possible solution thanks #2018-05-08 21:23:31 alexmiller if you want to write code instead, you could drop down to the tools.deps.alpha layer and write a few lines of code instead. #2018-05-09 00:34:46 richiardiandrea yeah I would really like to explore tools.deps.alpha a bit more, expecially the resolver part #2018-05-09 00:35:00 richiardiandrea one day... #2018-05-09 13:05:13 dominicm @alexmiller Can you think of a case where clj on a deps.edn file wouldn't resolve a dependency that tools.deps.alpha would resolve against a deps.edn file?
In particular, https://github.com/juxt/pack.alpha/issues/7#issue-321346647 something is going off to another repo. pack only considers the deps.edn, and doesn't merge the system one. For some reason it works if I add maven central, but what does that have to do with glassfish? #2018-05-09 13:05:47 alexmiller the install deps.edn includes the central and clojars maven repos #2018-05-09 13:07:17 alexmiller other than that, shouldn’t be any different #2018-05-09 13:09:30 dominicm @alexmiller I guess another way of describing this would be:
- I run clj against a deps.edn file - so everything should end up in .m2
- I run tools.deps.alpha against the same deps.edn and it fails as more needing more resolving #2018-05-09 13:09:51 alexmiller doesn’t make sense to me #2018-05-09 13:10:28 dominicm If I add the :mvn/repos to deps.edn, somehow it allows it to fetch from glassfish repo #2018-05-09 13:11:23 dominicm I'm not really sure what is going on here, and why this either needs to run every time (uncacheable) or doesn't get picked up by clj #2018-05-09 13:12:03 alexmiller I guess because once it’s in central they want you to get it from there? #2018-05-09 13:12:27 alexmiller dunno, weird #2018-05-09 13:12:40 dominicm yeah, real weird. #2018-05-09 13:13:04 dominicm This definitely highlights that I need a good way to do -Srepro without the local deps.edn file ever being picked up though 😉 #2018-05-09 13:13:59 dominicm Or rather, I need a way to pick up only the system deps.edn #2018-05-10 20:59:09 rickmoynihan @alexmiller: I’m bike shedding a little but what would you recommend calling the key for lein-tools-deps config. Currently it’s called :tools/deps which feels a little wrong, should it be :lein.tools/deps or :lein.tools.deps, or just :lein-tools-deps? #2018-05-10 21:02:09 rickmoynihan This is the key for using in a project.clj for configuring how the lein-tools-deps plugin should work with tools.deps #2018-05-10 21:03:19 alexmiller should be namespaced #2018-05-10 21:03:34 alexmiller wait, this is in project.clj? #2018-05-10 21:04:14 alexmiller I guess most stuff there is not namespaced #2018-05-10 21:04:24 alexmiller saying “lein” seems redundant #2018-05-10 21:04:59 alexmiller I guess I’d say :lein-tools-deps or :tools-deps #2018-05-10 21:05:56 dominicm No harm in setting a new precedent. #2018-05-10 21:06:00 rickmoynihan yeah its in project.clj #2018-05-10 21:06:08 rickmoynihan but thought it might be nice to set a precedent #2018-05-10 21:06:34 alexmiller :lein-tools-deps/config #2018-05-10 21:09:08 rickmoynihan yeah I had thought of that one too… I like that it nests config under it, but it gets wordy… though that’s not really an issue #2018-05-10 21:09:55 rickmoynihan it also matches the actual project name on github, and root namespace name #2018-05-10 21:10:04 rickmoynihan I think that’s definitely the most correct one #2018-05-10 21:10:06 rickmoynihan thanks #2018-05-11 14:28:16 ghadi I'm having trouble configuring an S3 private repo. To debug I've set -Srepro so that mavencentral & clojars are not seen by t.d.a, and I've set :mvn/repos to my own. For some reason I still get the error: cannot fetch the $dep from "central" #2018-05-11 14:33:12 ghadi never mind -- it's a bad error message. when all of the configured repos fail, it throws with one of them. #2018-05-11 14:43:48 ghadi Two other misc issues:
cannot use override-deps to override from a git/sha to a local/root
:mvn/repos doesn't work transitively when pointing to a :local/root #2018-05-11 15:04:31 alexmiller -Srepro does actually include the install deps.edn btw #2018-05-11 15:05:14 alexmiller at the moment that is #2018-05-11 15:07:14 alexmiller re “cannot use override-deps to override from a git/sha to a local/root” - with same lib name? prob worth a ticket if so #2018-05-11 15:07:22 ghadi Yes, same lib name #2018-05-11 15:07:25 ghadi 👌:skin-tone-4: #2018-05-11 15:07:32 alexmiller re “:mvn/repos doesn’t work transitively when pointing to a :local/root” - I don’t understand what that means #2018-05-11 15:08:21 ghadi A => :local/root B => B has different :mvn/repos => C #2018-05-11 15:09:24 alexmiller ok. that’s related to some of the other local dep issues that are already logged #2018-05-11 15:12:16 alexmiller @robert-stuttaford re twitter convo… current hack is:
{:deps {com.datomic/datomic.pro {:local/root "/path/to/datomic-pro-whatev.jar"}
com.datomic/datomic.pro.deps {:local/root "/path/to/dir-with-datomic-pom"}} #2018-05-11 15:12:59 alexmiller where the first pulls the jar and second treats the directory holding the pom file (which has to be named “pom.xml” as a local project to pick up the transitive deps) #2018-05-11 15:13:09 alexmiller that’s the hack #2018-05-11 15:13:32 alexmiller however, I just landed a commit to make local jar deps use their embedded pom file to handle this without the hack #2018-05-11 15:15:01 alexmiller will be in the next release #2018-05-11 15:13:38 ghadi (I don't think the transitive :mvn/repo thing should be supported, fwiw. Just logging the papercut) #2018-05-11 15:14:11 alexmiller well then don’t log it :) #2018-05-11 15:14:34 alexmiller I’m kind of on the fence though - the equivalent will work if you’re using maven deps I think #2018-05-11 15:14:35 ghadi I'll file a ticket for the git -> mvn override-dep #2018-05-11 17:44:01 ghadi yup, perfect #2018-05-12 11:06:39 robert-stuttaford thank you, @alexmiller! this is rad! #2018-05-12 12:08:56 mhuebert is there a simple way to specify a particular version of tools.deps.alpha itself (eg. to use last night’s release before it shows up on brew, or to point to a particular commit)? #2018-05-12 12:10:36 alexmiller No, but I just did a release last night and there is a pending brew pr which should get merged today #2018-05-12 12:11:08 dominicm time to bump packages #2018-05-12 12:11:53 alexmiller New tools.deps should be available #2018-05-12 12:13:23 alexmiller 0.5.435 #2018-05-12 13:08:39 mhuebert ah, brew has a nice way of pulling a PR. brew pull 27778 && brew upgrade clojure #2018-05-12 17:39:16 rickmoynihan I did consider not making it… as you’re probably a lein user if you’re reading it… But I figured new users might have been advised to use just one of the build tools; and be seeing this page, so thought it worth mentioning there are other choices too #2018-05-12 17:39:58 rickmoynihan or are you not a fan of boot? And would prefer everyone chose leiningen? 🙂 #2018-05-12 17:40:34 dominicm The opposite 😉 I'm the boot advocate at JUXT. #2018-05-12 17:40:57 rickmoynihan lol #2018-05-12 17:41:37 rickmoynihan then is my comment fair? #2018-05-12 17:48:44 dominicm I think it is, other than perhaps:
> Leiningen projects are harder to turn into unique snowflakes, which might be better or worse for you.
For me the "snowflake" term has gained a disparaging connotation. But I understand what you're getting at, and I think it's mostly fair. We had issues with lein because it couldn't build sass & cljs at the same time, which made it a pain for users jacking in, but maybe we're in snowflake territory there? #2018-05-12 18:06:42 rickmoynihan Yeah it is a slightly disparaging term; but I think it’s fair here - if you have a boot project it will often be pretty different to the next boot project; right?
More so than with lein projects.
I take your point that building sass/cljs simultaneously shouldn’t be such a unique a thing; but currently I think it is 😞 And yes if you want to do that boot is your best option; unless you want to build a lein plugin that essentially copies two other plugins entirely to compose them together on two futures 😞 #2018-05-13 14:52:01 alexmiller Now available:
- tools.deps.alpha 0.5.435
- clj 1.9.0.381
- Support for Maven deps in authenticated repositories (TDEPS-9 - thanks Dominic Monroe!)
- In local jar dependencies like {:local/root “path/to.jar”}, if the jar has an embedded pom, it will be used to traverse and include transitive deps
- Use exec for final Java invocation in scripts (TDEPS-76 - thanks Mikhail Gusarov!)
- Convey lib map via Java system property - a first step towards add-lib and other future functionality
For Maven deps in authenticated repositories, existing Maven infrastructure is used to convey credentials via the ~/.m2/settings.xml:
<servers>
…
<server>
<id>my-auth-repo</id>
<username>zango</username>
<password>123</password>
</server>
…
</servers>
Then in your deps.edn include a repo with a name matching the <id>:
{:deps
{authenticated/dep {:mvn/version "1.2.3"}}
:mvn/repos
{"my-auth-repo" {:url ""}}} #2018-05-13 22:11:06 zentrope Are these release-notes/change-logs recorded anywhere? I see new homebrew versions every now and then and wonder what the changes are. #2018-05-13 22:11:52 zentrope In other words, release notes of the whole thing CLI utility, not just individual libs? #2018-05-13 23:34:50 seancorfield @zentrope ^ does that help? #2018-05-13 23:40:45 zentrope @seancorfield Yep! Thanks. #2018-05-13 23:41:52 zentrope I wish brew packages came with a change log of some sort built-in. Or brew info would show the latest changes. Mostly I don’t care, but sometimes it’s interesting. #2018-05-14 00:25:56 dottedmag @zentrope brew log -p clojure #2018-05-14 00:26:39 alexmiller Both of those GitHub projects also have a CHANGELOG file too #2018-05-14 00:28:13 alexmiller The scripts do install a man page for clj and clojure - I could probably add something there but I don’t know if that’s weird #2018-05-14 00:28:25 zentrope Yes, I saw those. I glanced right over brew-install as a project before asking. #2018-05-14 00:29:39 zentrope Yeah, easy change logs aren’t really part of the brew thing, it seems, outside of that @dottedmag just showed. #2018-05-14 00:55:04 alexmiller I added links from the man page to the changelogs for the next release #2018-05-15 09:05:03 shakdwipeea Hi guys, cross posting from #clojure
I am getting a dependency mismatch with s3. In my project I have specified dependency on com.amazonaws/aws-java-sdk-s3 {:mvn/version "1.11.261"}.
Also, I have a library whose transitive dependency include a different version of sdk . I have specified a exclusion for s3 as:
libB {:mvn/version "2.0.0-SNAPSHOT"
:exclusions [com.amazonaws/aws-java-sdk-s3]}
Now, when I do clj -Stree. I can see that only the correct version is being included.
But when I tried to find the location of sdk being included using
System.out.println(SDKGlobalConfiguration.class.getProtectionDomain().getCodeSource().getLocation()); as per https://github.com/aws/aws-sdk-java/issues/1071
I see that it is referring to the location of the jar of the transitive dependency which is the older version of aws sdk.
FWIW I am using the clj tool and deps.edn and the conflicting libraries are java based.
I am able to use the correct version from boot for the same tree. #2018-05-15 22:29:06 kenny Where is the code for the clj cli tool located? #2018-05-15 22:31:16 ghadi clj == rlwrap clojure basically #2018-05-15 22:39:34 kenny Thanks #2018-05-16 17:45:03 kenny Is there a way to print the final deps.edn as data to stdout? #2018-05-16 18:14:44 alexmiller currently, no #2018-05-16 18:19:34 alexmiller you could use the output of -Sdescribe + a few calls in the api to replicate it #2018-05-16 18:23:03 kenny Cool, figured it out. That option would be useful 🙂 #2018-05-16 18:27:28 kenny For those interested... Some psuedo code:
(defn full-deps-edn
[]
(let [config-files (:config-files (exec-sync-edn "clj -Sdescribe"))
deps "'{:deps {org.clojure/tools.deps.alpha {:mvn/version \"0.5.435\"}}}'"
code (str/join " " ['(require '[clojure.tools.deps.alpha.reader :as reader])
(list 'reader/read-deps config-files)])
cmd (str "clj -Sdeps " deps " -e '" code "'")]
(exec-sync-edn cmd)))
#2018-05-17 22:06:57 kenny How do git deps work in a monolith type repo? It looks like I can only specify the root git URL. #2018-05-17 22:16:05 alexmiller You can also use :deps/root to specify a relative dir path from the root #2018-05-17 22:17:45 alexmiller Yeah those are a bit lagging #2018-05-18 20:53:42 kenny Is there a reason you shouldn't always use the -A CLI option? #2018-05-18 20:55:43 dominicm if the test alias uses test-runner, you may not want the main aspect of it 🙂 #2018-05-19 00:14:23 seancorfield This is why I think it's better practice to have separate test and runner aliases 🙂 #2018-05-18 20:57:53 kenny Could you clarify what you mean? What is test-runner? And what is the main aspect of it? #2018-05-18 20:58:17 dominicm Sorry, I should be clearer, you may not want the :main-opts option to kick in. #2018-05-18 20:59:14 kenny Ah that makes sense. Is that the only reason? #2018-05-18 22:01:55 hagmonk Fun! clojure -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.6.0"} gorilla-repl {:mvn/version "0.4.0"}}}' -m gorilla-repl.core #2018-05-20 13:11:44 raymcdermott I’m trying to make use of add-lib using clojure 1.10-alpha4 with the prepl … #2018-05-20 13:12:06 raymcdermott @alexmiller is that combo possible? #2018-05-20 13:12:21 dominicm @raymcdermott what happened when you tried? #2018-05-20 13:16:19 raymcdermott :cause “Context classloader is not a DynamicClassLoader” #2018-05-20 13:17:19 raymcdermott bit weird though cos it partly works #2018-05-20 13:20:33 raymcdermott so it ‘works’ when evaluating something that is in the path … but that’s not so useful 😉 #2018-05-20 13:24:18 raymcdermott but I’m stabbing and poking around, so asking here to get a steer on whether it’s meant to be viable #2018-05-20 13:44:34 dominicm @raymcdermott My suspicion is that however the socket repl starts, doesn't cause a DynamicClassLoader to be run, I think you can insert one, but can't remember from the top of my head how that's done. #2018-05-20 13:46:12 alexmiller That would be a good guess but I haven’t tried it #2018-05-20 13:52:14 dominicm @raymcdermott How are you running that clj file? #2018-05-20 13:52:25 raymcdermott lein run #2018-05-20 13:53:18 dominicm I think that uses aot-ish stuff, but I'm not certain. #2018-05-20 13:55:38 raymcdermott I’ll run it with clj and see if it makes any difference #2018-05-20 13:57:16 mfikes See that bit where it forced the issue so that the classloader is a clojure.lang.DynamicClassLoader. This might not be the "correct" solution, but it worked. #2018-05-20 13:58:37 raymcdermott nice - I’ll check it out #2018-05-20 14:23:57 alexmiller Setting the context classloader to a DCL should work #2018-05-20 14:24:28 alexmiller Although that is a per-thread solution #2018-05-20 14:26:08 raymcdermott ok, so it’s the same whether I run it via CLJ or not … just thought I would check that out first #2018-05-20 14:26:39 raymcdermott DCL next - one thread is fine for now ( who needs more! 😉 ) #2018-05-20 16:01:03 alexmiller wow, old school reference :) #2018-05-20 14:41:02 raymcdermott party-corgi #2018-05-20 14:41:12 raymcdermott with DCL all works #2018-05-20 14:41:52 raymcdermott good call @mfikes, will do - had it in the REPL code but will move it further out #2018-05-20 15:49:05 raymcdermott and - bonus - my code is now all set up with deps.edn and running with CLJ 😎 #2018-05-20 16:30:33 robert-stuttaford what’s the correct way to force a SNAPSHOT download with clj / tools.deps? #2018-05-20 17:10:43 alexmiller There isn’t a way at the moment - I have a ticket for that. You can just delete the version from your .m2 though and use -Sforce #2018-05-20 18:04:53 robert-stuttaford thanks Alex! #2018-05-21 10:34:36 robert-stuttaford @alexmiller @dominicm thanks for TDEPS-9 - using Datomic PRO with clj is rad #2018-05-21 10:35:22 dominicm @robert-stuttaford I setup the JUXT website last night, was nice 🙂 #2018-05-21 10:35:50 robert-stuttaford i recognise this may be a daft question, but is there perhaps a way to run two ‘mains’ at once from clj ? i see the docs say that it’ll run the last one encountered, but i’m hoping to get a boot-like thing going where i can run both figwheel-main and my 10-liner CIDER repl script in one process #2018-05-21 10:36:16 dominicm @robert-stuttaford Update to cider 0.17 and use -e before your normal -m #2018-05-21 10:36:33 robert-stuttaford -reads about -e- #2018-05-21 10:37:02 dominicm ^^ this might be useful to see what cider generates when it does jack-in, and hard-code it in 🙂 #2018-05-21 10:37:17 dominicm but you could implement cider-nrepl.main/init yourself. #2018-05-21 10:37:54 robert-stuttaford i’ve got a working cider script:
(require '[clojure.tools.nrepl.server :refer [start-server]]
'[refactor-nrepl.middleware :refer [wrap-refactor]])
(defn nrepl-handler []
(require 'cider.nrepl)
(ns-resolve 'cider.nrepl 'cider-nrepl-handler))
(def port 7800)
(spit ".nrepl-port" port)
(start-server :port port :handler (wrap-refactor (nrepl-handler)))
(println "Started nREPL on port" port)
(.addShutdownHook (Runtime/getRuntime)
(Thread. #( ".nrepl-port")))
#2018-05-21 10:38:14 robert-stuttaford i want to run it and figwheel-main together, in one process. right now i use two #2018-05-21 10:38:28 dominicm Are you using cljs? That doesn't look like it will work with cljs. Unless you have a separate nrepl for figwheel? #2018-05-21 10:39:22 robert-stuttaford hmm, i guess there’s no way around it; i’ll have to write a clj script that starts the nrepl-server and invokes the figwheel-main stuff afterwards #2018-05-21 10:40:30 dominicm You can replace that whole file with:
clj -e "(require 'cider-nrepl.main) (cider-nrepl.main/init '[refactor-nrepl.middleware/wrap-refactor])"
#2018-05-21 10:40:43 dominicm I'm presuming that can go into the deps.edn somewhere. #2018-05-21 10:45:04 dominicm @robert-stuttaford Apparently not, but mains can run other mains.
(defn -main [& args]
(start-nrepl)
(apply clojure.repl/main args))
#2018-05-21 12:07:28 robert-stuttaford turns out the figwheel-main api is super simple, this was a total non-issue #2018-05-21 12:07:43 robert-stuttaford ;; nREPL
(require '[clojure.tools.nrepl.server :refer [start-server]]
'[refactor-nrepl.middleware :refer [wrap-refactor]])
(defn nrepl-handler []
(require 'cider.nrepl)
(ns-resolve 'cider.nrepl 'cider-nrepl-handler))
(def port 7800)
(spit ".nrepl-port" port)
(start-server :port port :handler (wrap-refactor (nrepl-handler)))
(println "Started nREPL on port" port)
(.addShutdownHook (Runtime/getRuntime)
(Thread. #( ".nrepl-port")))
;; Figwheel
(require '[figwheel.main :refer [start]])
(start "dev")
#2018-05-21 19:14:58 jeff.terrell With the new clojure CLI tool, can I have a standalone .clj script, i.e. apart from a src/ directory and without a corresponding deps.edn file, ideally invokable via a simple shebang line (`#!`) at the top of the file? If so, what would that shebang line be? #2018-05-21 19:16:15 dominicm Yes #2018-05-21 19:16:44 dominicm Digging it up now #2018-05-21 19:20:16 dominicm @jeff.terrell
#!/bin/sh
"exec" "clojure" "-Sdeps" '{:deps {funcool/tubax {:mvn/version "0.2.0"}}}' "$0" " #2018-05-21 19:20:55 jeff.terrell Impressive. Thanks, @dominicm! I'm assuming the :deps are optional? #2018-05-21 19:22:17 dominicm @jeff.terrell what do you mean optional? #2018-05-21 19:22:43 jeff.terrell As in, I don't need any deps. I can just leave that part out, right? #2018-05-21 19:23:32 dominicm Oh, sure. In that case you can just do.
/usr/bin/env clojure
...
#2018-05-21 19:24:02 dominicm No args = shebang. Args = exec #2018-05-21 19:25:40 jeff.terrell Ah perfect. I did that initially, but I was getting confused because I expected clojure to automatically call my -main function. But I see now that it was working fine. Thanks for the help! #2018-05-21 19:31:21 dominicm @jeff.terrell I came up with it. #2018-05-21 19:47:35 jeff.terrell Nice trick! Thanks again. #2018-05-21 21:33:29 jeff.terrell OK, followup question: how do I access command-line arguments to a #!/usr/bin/env clojure standalone script? #2018-05-21 21:46:36 mfikes @jeff.terrell They will be in *command-line-args* #2018-05-21 21:47:36 jeff.terrell That's it. Thanks! #2018-05-22 14:04:48 borkdude I credited Planck for it. #2018-05-22 15:02:00 jeff.terrell That is one freaky shell/Clojure chimera there, @borkdude. simple_smile Well done. #2018-05-22 15:05:36 jeff.terrell I think you should be able to replace the outer double-quotes after the semi-colons with single-quotes, so that you don't have to escape the inner ones. #2018-05-22 15:06:13 jeff.terrell (Bash deals with single quotes, and Clojure isn't going to care about them after the ;. That's my reasoning, anyway, but maybe I'm missing something.) #2018-05-22 15:12:18 alexmiller boot embeds a jar inside the shell script. I think that wins the chimera game for me. #2018-05-22 15:13:36 borkdude @jeff.terrell thanks, made the change #2018-05-22 15:13:43 mfikes That one wins the "Hello my Baby" frog jumping out of the box award #2018-05-23 07:16:51 martinklepsch The error message when deps.edn’s parens aren’t balanced could be friendlier:
Error building classpath. EOF while reading
java.lang.RuntimeException: EOF while reading
#2018-05-23 09:41:43 alexmiller File a ticket #2018-05-23 14:49:52 kenny lein-tools-deps does not seem to be marking my src directory as a source root. I am using [lein-tools-deps "0.3.0-SNAPSHOT"] with :tools/deps set to [:system :home "/path/to/deps.edn"]. Has anyone else ran into this issue? #2018-05-23 15:09:26 mhuebert @kenny I have hit this just in the last couple of days, setting up a monorepo with a bunch of directories which each have their own deps.edn. I ended up adding "src" to :paths in each deps.edn #2018-05-23 15:12:20 kenny @mhuebert That is my exact situation and that solution did not work for me 😞 #2018-05-23 15:16:53 mhuebert hmm, I am trying to set up the repo on another machine now to do some more testing and getting a completely different error with lein-tools-deps
mattpro:chia MattPro$ lein pom
java.lang.NullPointerException: null
at clojure.tools.deps.alpha.util.maven$make_session.invokeStatic (maven.clj:105)
clojure.tools.deps.alpha.util.maven$make_session.invoke (maven.clj:102)
clojure.tools.deps.alpha.extensions.maven$eval851$fn__853.invoke (maven.clj:74)
clojure.lang.MultiFn.invoke (MultiFn.java:243)
clojure.tools.deps.alpha$expand_deps.invokeStatic (alpha.clj:170)
clojure.tools.deps.alpha$expand_deps.invoke (alpha.clj:152)
clojure.tools.deps.alpha$resolve_deps.invokeStatic (alpha.clj:215)
clojure.tools.deps.alpha$resolve_deps.invoke (alpha.clj:197)
lein_tools_deps.plugin$resolve_deps.invokeStatic (plugin.clj:78)
lein_tools_deps.plugin$resolve_deps.invoke (plugin.clj:69)
lein_tools_deps.plugin$middleware.invokeStatic (plugin.clj:93)
lein_tools_deps.plugin$middleware.invoke (plugin.clj:85)
#2018-05-23 16:05:24 mhuebert *figured out this was due to a too-old Leiningen version #2018-05-23 15:19:28 mhuebert the line numbers on make_session make me think it is somehow not a recent version of tools.deps, though i just ran brew upgrade clojure #2018-05-23 15:22:22 alexmiller this is a known issue with interpreting the paths for dependent repos #2018-05-23 15:23:30 alexmiller or at least that’s my guess as to what you’re seeing #2018-05-23 15:23:56 alexmiller setting :paths explicitly in each deps.edn should workaround it #2018-05-23 15:25:24 mhuebert that workaround worked in my case #2018-05-23 15:26:11 mhuebert are local paths skipped intentionally during pom-generation or is that just WIP? #2018-05-23 15:27:55 mhuebert am doing a workaround currently, generating a project.clj for lein to use lein-tools-deps to generate a full pom which includes everything necessary for IntelliJ to figure out the project #2018-05-23 15:34:31 alexmiller the first local path is added as the build source path, but I think that actually needs to be changed so it’s a resource path. I don’t think there’s actually a ticket right now about that #2018-05-23 15:35:52 alexmiller if you can file a ticket to remind me (or a patch!) that would be helpful #2018-05-23 15:42:04 mhuebert hmm. why would it be a resource path? #2018-05-23 15:42:54 mhuebert I am comparing to what lein generates, it also makes <build><sourceDirectory>src</sourceDirectory> ... </build> #2018-05-23 15:43:06 mhuebert that part seems to play well with intellij #2018-05-23 15:46:29 mhuebert Skipping paths: _build
Skipping coordinate: #:local{:root reactive}
Skipping coordinate: #:local{:root triple-db}
Skipping coordinate: #:local{:root view}
Skipping coordinate: #:local{:root gesture}
Skipping coordinate: #:local{:root style}
#2018-05-23 15:48:49 mhuebert I am wondering what can/should done about ^^these sorts of paths & local deps #2018-05-23 15:50:48 alexmiller oh, there is nothing that can be done about them - Maven has no concept of local deps #2018-05-23 15:51:05 alexmiller or git deps #2018-05-23 15:51:34 alexmiller so that’s never going to work #2018-05-23 15:52:05 alexmiller I guess you could write a custom Maven wagon and use that, but I’m not going to do that #2018-05-23 15:57:50 mhuebert it looks like lein pom uses a plugin (`build-helper-maven-plugin`) to add the local sources for those:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>_build</source>
<source>reactive/src</source>
<source>triple-db/src</source>
<source>view/src</source>
<source>view/test</source>
<source>gesture/src</source>
<source>style/src</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
#2018-05-23 16:01:11 alexmiller yeah, I’m not going to use that #2018-05-23 16:01:44 alexmiller you can add these to your pom manually if you like. -Spom will sync an existing pom, just updating the rest of it #2018-05-23 16:03:21 mhuebert ok. at some point maybe intelliJ/cursive will support deps.edn directly #2018-05-23 16:47:31 alexmiller I think he’s started on that #2018-05-24 08:59:58 mhuebert @kenny I just found something that may be related to your issues yesterday. In my mono-repo-of-`:local/root`-projects, if I change one of the deps.edn files inside of a :local/root directory, tools.deps does not recognize the change, and uses its previously cached classpath. I have to make some minor modification to the root deps.edn file (like, add a space somewhere) and then it invalidates the cache. So you may have configured everything correctly and clj was just using an old classpath. Can test for this by running clj -Spath before/after changes to see when it uses the cache vs. when it generates a fresh one. #2018-05-24 09:08:06 mhuebert @alexmiller would it be worth creating an issue for this? I am not sure exactly where/how cache invalidation is handled #2018-05-24 12:12:00 alexmiller I have an issue for it #2018-05-24 12:15:40 alexmiller Actually looks like maybe I never filed a public issue but I have it in my internal list #2018-05-24 12:15:50 alexmiller Workaround is to use -Sforce #2018-05-24 15:54:08 borkdude -e, --eval string Eval exprs in string; print non-nil values
is it possible to turn off the printing? #2018-05-24 16:02:18 borkdude never mind #2018-05-24 18:47:08 raymcdermott when using add-lib I sometimes get errors when adding via a git SHA …. Manifest type not detected when finding deps #2018-05-24 18:47:42 raymcdermott here is the longer stack trace #2018-05-24 18:48:11 ghadi raymcdermott you may be add-lib'bing a git repo that has no deps.edn in it #2018-05-24 18:48:35 raymcdermott a ha! #2018-05-24 18:48:54 raymcdermott @ghadi I did not realize that was a pre-requisite #2018-05-24 18:49:08 raymcdermott pretty sure that is the case then #2018-05-24 18:49:49 ghadi Yeah. tools.deps cannot infer third-party dependencies without some sort of manifest file #2018-05-24 18:50:01 ghadi right now deps.edn and pom.xml are supported #2018-05-24 18:50:19 raymcdermott ok, cool so that’s why Maven deps are safer #2018-05-24 18:50:36 raymcdermott for the time being at least 😉 #2018-05-24 18:52:32 raymcdermott at least I know which repos I can demo now #2018-05-24 18:55:25 raymcdermott off the wall question … has tools.deps been tried out on graal? #2018-05-24 18:58:50 alexmiller Not to my knowledge #2018-05-24 18:59:04 alexmiller I suspect it’s bounded by network io usually #2018-05-24 19:00:13 raymcdermott I was thinking that it probably couldn’t change the classpath #2018-05-24 19:01:12 ghadi yeah, it cannot. add-lib will not work over there.
Does the REPL work in Graal? #2018-05-24 19:01:23 raymcdermott I’m guessing that an app can still expose a socket-server #2018-05-24 19:02:11 ghadi (I'm assuming you mean the Graal AOT binary part, not just running Graal through JVMCI) #2018-05-24 19:02:44 ghadi the repl (whether socket server or otherwise) generates classes, which is incompatible with Graal's AOT native images. #2018-05-24 19:03:07 ghadi But all that should work properly through Graal non-AOT (the JVMCI) #2018-05-24 19:03:44 raymcdermott that sounds legit… #2018-05-24 19:04:05 raymcdermott so that means - in theory at least - that you can call another language via the REPL #2018-05-24 19:07:28 raymcdermott they have a polyglot shell so I’m guessing yes at this stage #2018-05-24 19:08:42 raymcdermott although it seems to switch languages rather than offer FFI #2018-05-24 19:09:39 raymcdermott hmmm this seems to be the thing #2018-05-24 19:10:44 raymcdermott ok anyway … that’s a different channel #2018-05-24 19:11:15 raymcdermott thanks for the help @ghadi #2018-05-24 19:51:30 raymcdermott btw I made a small demo app with the pre-reqs in place for add-lib #2018-05-24 19:52:10 raymcdermott I have proven it with my REPL so it might work elsewhere 🙂 #2018-05-25 18:12:28 kenny Is there a reason the -Spom option does not offer any way to configure the generated pom (i.e. project name and group id are the directory name and the version is hardcoded to 0.1.0)? #2018-05-25 18:15:05 alexmiller -Spom syncs an existing pom, so run it, then modify the pom. subsequent runs will keep that stuff. #2018-05-25 19:46:09 kenny If I run this
clj -Srepro -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.9.0"}}}' -Spath
In a directory containing a deps.edn that looks like this:
{:deps {org.clojure/clojure {:mvn/version "1.8.0"}
org.clojure/core.async {:mvn/version "0.4.474"}}}
I get a classpath containing core.async. Why does that happen? I thought -Sdeps was supposed to be the final deps.edn file. #2018-05-25 20:04:11 hiredman it looks like dep.edns are merged, and -Sdeps is the final value merged #2018-05-25 20:05:17 kenny Oh, you're right. Hmm, ok. It'd be nice to have a way to easily reproduce issues by explicitly passing in the one and only deps edn. #2018-05-25 21:41:38 seancorfield There's an argument with -Srepro that it should ignore everything except the current directory's deps.edn -- on the grounds that it would make a project "reproducible". As it is, ignoring the install directory with -Srepro means that you need extra stuff in your project deps.edn that you wouldn't normally need (`:paths`, repos?) which means you can't just take an arbitrary project's deps.edn file and run it with -Srepro unless that's how the author expected it to be run. #2018-05-25 21:47:58 kenny I was asking for a directory independent way of running clj. That topic is somewhat related but still different. #2018-05-25 21:56:50 seancorfield Yeah, having more control over which of the four deps locations were consulted would be very nice. #2018-05-25 21:57:25 seancorfield And your scenario -- ignoring the current directory's deps.edn -- is a good use case that it would be nice to see supported... #2018-05-26 15:41:43 renewdoit Is there any idea to download deps in parallel to speed up? #2018-05-26 16:59:01 alexmiller It’s something I have thought about. It’s a little tricky the way things are set up right now but def doable #2018-05-28 09:01:47 borkdude without Srepro deps aren’t downloaded but with it they are?
clojure -Srepro -Spath
I need the path but also the effect of deps being downloaded, but nothing else. What’s the best option for this? #2018-05-28 09:02:36 borkdude this is excellent for lumo/planck btw, very fast #2018-05-28 13:43:30 alexmiller repro should not affect downloading. Are you just seeing m2 caching? #2018-05-28 13:58:24 borkdude @alexmiller when I e.g. delete tools.cli from my .m2 repo, only -Spath does not force downloading, but it works with -Srepro. What should the behavior be? #2018-05-28 14:07:00 alexmiller Oh, you’re misinterpreting stuff due to classpath caching #2018-05-28 14:08:38 alexmiller Normally, if there is a cached classpath, it’s used (and the actual paths aren’t checked, so it won’t notice you deleted from m2). But if there is no cached classpath it will download #2018-05-28 14:08:45 borkdude if I can set M2_HOME somehow, I could try to reproduce. Don’t know how to do that with clj #2018-05-28 14:09:03 borkdude ah, that explains it #2018-05-28 14:09:04 alexmiller With repro, I think it does not use the cached classpath #2018-05-28 14:09:12 alexmiller So forces download #2018-05-28 14:09:41 alexmiller You can use -Spath -Sforce to do what you want #2018-05-28 14:09:58 borkdude so in normal circumstances ( I don’t delete something from .m2 manually ) all should work according to plan #2018-05-28 14:10:49 alexmiller Yes, in the normal circumstances things work normally #2018-05-28 14:11:04 alexmiller ;) #2018-05-28 14:11:19 borkdude yeah, but -Sforce takes longer, that’s not preferred for planck or lumo, so I’ll go for just -Spath #2018-05-28 14:11:38 alexmiller We actually have a ticket open for noticing broken paths #2018-05-28 14:11:40 borkdude hehe, yes, I mean, there is no logic to actually check for the existence of jars after classpath caching #2018-05-28 14:12:00 borkdude so, I understand now, thanks #2018-05-28 14:12:26 borkdude side question, is there a way to change which .m2/repository to use with clj? #2018-05-29 16:55:48 rickmoynihan Merely as an extra data point I ran into this issue yesterday because I’d blown an .m2 dep away whilst deving/testing some CI stuff. I realised I also needed to blow .cpcache away too to get the effect I wanted; but either a good message, or re-resolving the missing deps would’ve been super nice. It could easily have been a hard error to diagnose. #2018-05-29 16:56:50 alexmiller yeah, I’ve been there (even knowing how it works) #2018-05-28 14:13:11 alexmiller Yes, you can set :deps/local-repo in deps.edn map #2018-05-28 14:13:30 borkdude ah, that’s actually the issue yes. well, if it takes extra time it would be worth considering not doing it… or with another option #2018-05-28 14:13:57 borkdude if it’s a couple of ms, it would be ok #2018-05-28 14:14:22 alexmiller Yeah I haven’t done any eval on it yet #2018-05-28 14:14:27 alexmiller TBD #2018-05-28 14:15:06 alexmiller There are also cases where you might expect to have a missing path dir #2018-05-28 14:15:32 alexmiller And would want that to be ok #2018-05-28 14:16:45 borkdude I have this in deps.edn:
{:deps/local-repo "/tmp/repro"}
and do this from /tmp/repro:
clj -Sforce -Sdeps '{:deps {org.clojure/tools.cli {:mvn/version "0.3.7"}}}' -e "(require '[clojure.tools.cli])"
it doesn’t seem to use the local dir #2018-05-28 14:28:59 borkdude just upgraded to #2018-05-28 14:29:10 borkdude also put everything inside the deps.edn file #2018-05-28 14:31:05 borkdude ah, it was :mvn/local-repo #2018-05-28 16:44:55 alexmiller Oh sorry, yeah that :) #2018-05-29 01:00:39 hagmonk Does anyone have a deps.edn snippet or advice on how to get a browser based CLJS app running, with an nREPL that Cider is able to reach? Basically like the new CLJS quickstart guide, except with the REPL in emacs instead. #2018-05-29 03:39:02 richiardiandrea @hagmonk the new cider should inject all the necessary deps as long as you have a deps.edn in the root folder #2018-05-29 03:39:18 richiardiandrea Also better ask this in #cider #2018-05-29 03:54:18 hagmonk @richiardiandrea to rephrase my question so that it's not being confused with something cider specific: does anyone have a deps.edn snippet or advice on how to get a browser based CLJS app running, using cljs.main, such that it starts an nREPL server rather than an interactive terminal REPL? #2018-05-29 07:52:16 delaguardo Hi there! I have a question about :default-deps. Does anybody have a good example how to use it? Official documentation tells it provides a set of default coordinate versions to use if no coordinate is specified. But how I can not specify coordinate versions for the :deps or :extra-deps? #2018-05-29 07:52:16 delaguardo Hi there! I have a question about :default-deps. Does anybody have a good example how to use it? Official documentation tells it provides a set of default coordinate versions to use if no coordinate is specified. But how I can not specify coordinate versions for the :deps or :extra-deps? #2018-05-29 11:54:42 alexmiller This area is a little incomplete and there is not really a great story around how to use it yet #2018-05-29 12:18:15 delaguardo There is my case if you are interested: I’m trying to create “monorepo” with root deps.edn and a few “sub-projects” with their own local deps.edn. I would like to have a list of all used dependencies with their coordinates as a :default-deps in root deps.edn. All local deps.edn files should have only a list of dependencies without coordinates specified. #2018-05-29 13:22:27 alexmiller yeah, this is in line with what we intended but there needs to be another round of thinking on how to make this a reality #2018-05-29 15:06:13 delaguardo 👍 #2018-05-29 10:53:09 shakdwipeea Hi guys, One of my transitive deps is trying to pull javax.activation and running into this error:
Error building classpath. Could not find artifact activation:activation:jar:1.0.2 in central ()
org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact activation:activation:jar:1.0.2 in central ()
Turns out the jar is not available in maven but it is available in one of the adobe public repos.
So I added
{:mvn/repos {"adobe-public" {:url ""}}}
Now this only works if javax.activation is a direct dependency but not when it is a transitive dep.
Also, the jar is available in maven local but I am still running into this. #2018-05-29 10:58:02 dominicm I've seen some funky poms where the pom actually refers to other repositories and those are transitively picked up. #2018-05-29 11:56:46 alexmiller @shakdwipeea are you getting to these transitive deps through a maven dep or a local/git dep? #2018-05-29 12:00:54 shakdwipeea @alexmiller It is coming through a git dep. I also tried with https://jitpack.io to use it as a mvn dep but that also didn't work. The funny thing is that the required dep is present in my maven local but still this fails. #2018-05-29 12:01:36 shakdwipeea FWIW the dep is a java project and uses maven. #2018-05-29 12:14:48 alexmiller Yeah this is a known issue with git and local deps not picking up the repos in the pom #2018-05-29 12:17:50 alexmiller Or at least I thought it was, I don’t see a ticket for it. I’d be happy to have a ticket with more specific details so I can take a look #2018-05-29 15:23:37 shakdwipeea I'll try to create a minimal repro and file a ticket. #2018-05-29 15:44:53 alexmiller thx #2018-05-30 07:05:08 shakdwipeea I tried to reproduce the issue on a new repo. https://github.com/shakdwipeea/tdeps-transitive-repo-test Now when you clj -A:test-git it fails with the message that
`
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact activation:activation:jar:1.0.2 in central ()
The same deps works with the boot file that is present in the repo. The interesting thing is that
when I do boot show -d, I get
[com.github.shakdwipeea/smtp "0.3.1"]
└── [javax.javaee/javaee "6.0-alpha-1"]
├── [javax.activation/activation "1.1-rev-1"]
...
This uses the version 1.1-rev-1 which is available in the maven repo.
The 1.0.2 version should not even be fetched.
I am not sure if I am doing sth wrong, so wanted to consult before opening a ticket. #2018-05-29 18:13:03 hagmonk @alexmiller how does one contribute to the tools.deps.alpha github wiki? I wanted to put a link to depify, and @seancorfield's clj-new. #2018-05-29 18:14:37 alexmiller it’s a wiki - anyone should be able to edit #2018-05-29 18:15:11 hagmonk Cool! I did see the edit button but didn't want to boldly assume :) #2018-05-29 18:17:15 seancorfield @hagmonk Thank you in advance for adding a link to clj-new 🙂 #2018-05-29 18:21:49 hagmonk Done! I'm sure there are more that could be added ... #2018-05-30 11:28:52 borkdude @shakdwipeea what if you move the deps to a toplevel :deps? #2018-05-30 11:29:58 shakdwipeea I had tried that.. Same result.. Made aliases to test out the two conditions separately #2018-05-31 16:03:21 kenny How do I load a Clojure file on the classpath from the CLI? I tried doing clj -i my.ns and I get a FileNotFound exception. #2018-05-31 16:05:37 kenny I guess I could just call load in -e. Is there an easier way though? #2018-05-31 16:07:16 ghadi clj -i foo.clj should work -i expects a file name, not a namespace #2018-05-31 16:22:54 kenny Say I have a monolith type git repo that looks something like this:
mylib1
mylib2
mylib2 has a deps.edn files that looks like this:
{:deps {mylib1 {:local/root "../mylib1"}}}
Is it possible to use mylib2 as a git dep (`{:git/url "https://github.com/..." :sha "..." :deps/root "mylib2"}`) from the CLI and have mylib1 dependency be resolved using the :local/root path and the same :sha used previously? #2018-05-31 16:28:17 kenny Or perhaps even changing mylib1 coordinate to be {:git/url "" :deps/root "mylib1"} and the :sha would be automatically set to the same :sha used for the the git URL ... that is already on the classpath. #2018-05-31 16:32:16 alexmiller In short, no - git libs are only resolved from git #2018-05-31 19:54:19 kenny If I have foo.clj:
(ns foo)
(defn bar
[& args]
(prn *command-line-args*))
And run:
clj -i foo.clj -e '(foo/bar)' asd
I get a FileNotFoundException. #2018-05-31 19:55:16 kenny I assume this is because it is expecting the last param to be a path. Why does it require that? #2018-05-31 19:58:22 ghadi yes the last param is a path #2018-05-31 19:59:21 kenny Right. Why does it require that? #2018-05-31 20:02:15 zentrope Are relative paths wonky for deps? For instance, if you local/root a deps.end which in turn local/root’s a dep (something like “../../../whatever”) is the “../../..” going to mess things up if the top level isn’t in a “../../../” relationship to that project? #2018-05-31 20:03:14 seancorfield @kenny Use - to introduce the arguments: clj -i foo.clj -e '(foo/bar)' - asd #2018-05-31 20:03:16 zentrope Hm. Overriding it in the “current” deps.edn solves it. #2018-05-31 20:04:09 kenny @seancorfield The program doesn't exit then. #2018-05-31 20:05:08 ghadi @kenny what are you trying to achieve? #2018-05-31 20:05:19 seancorfield Ah, yes, you'd have to send it empty input 🙂 #2018-05-31 20:05:25 kenny To evaluate the function in -e. #2018-05-31 20:05:26 ghadi there may be a more straighforward path. #2018-05-31 20:05:35 kenny (foo/bar) #2018-05-31 20:06:02 seancorfield You can only get *command-line-args* if you have main-opts #2018-05-31 20:06:25 ghadi you can -e "(require 'foo)" -e "(foo/bar)" #2018-05-31 20:07:18 kenny I also want *command-line-args* 🙂 #2018-05-31 20:07:23 ghadi oh -- yeah you can't. #2018-05-31 20:08:10 ghadi Or, this (cue eye roll):
clj - <<EOM
(require 'foo)
(foo/bar)
EOM
#2018-05-31 20:08:23 ghadi (In bash or zsh) #2018-05-31 20:08:24 seancorfield Rename bar to -main and then clj -i foo.clj -m foo asd #2018-05-31 20:08:28 ghadi ^^^ #2018-05-31 20:08:50 seancorfield (! 633)-> cat foo.clj
(ns foo)
(defn -main
[& args]
(prn *command-line-args*))
Thu May 31 13:08:39
(sean)-(jobs:0)-(~/clojure/kenny)
(! 634)-> clj -i foo.clj -m foo asd
("asd")
#2018-05-31 20:08:53 kenny Bleh. And I can't rename because -main is used for something else. #2018-05-31 20:09:08 kenny I also may want to call different functions besides -main. #2018-05-31 20:09:40 seancorfield Add another namespace for this entry point? #2018-05-31 20:10:21 kenny Perhaps. But I may have 50 different functions that I'd like to call. I don't want 50 namespaces to simply call a function. #2018-05-31 20:11:21 seancorfield OK, how about this: (! 638)-> cat foo.clj
(ns foo)
(defn -main [& args] (println "I am foo's -main"))
(ns foo.main)
(defn -main
[& args]
(prn *command-line-args*))
Thu May 31 13:10:56
(sean)-(jobs:0)-(~/clojure/kenny)
(! 639)-> clj -i foo.clj -m foo
I am foo's -main
Thu May 31 13:11:04
(sean)-(jobs:0)-(~/clojure/kenny)
(! 640)-> clj -i foo.clj -m foo.main asd
("asd")
#2018-05-31 20:11:31 seancorfield (multiple namespaces in one file 🙂 ) #2018-05-31 20:12:03 kenny Ah, interesting. #2018-05-31 20:12:44 kenny Still feels like a bit of a hack but that could work. It seems like it'd be useful to have a CLI option to execute a function besides -main. #2018-05-31 20:13:51 seancorfield We have a job script at work whose first argument is a namespace-qualified symbol and it calls that from -main. So you'd say clj -i job.clj -m job job/foo asd #2018-05-31 20:14:07 seancorfield Relying on dynamic require and resolve #2018-05-31 20:14:42 kenny Yeah, I think I like that option more. #2018-05-31 21:43:17 kenny Should :override-deps work to change an artifact's coordinate from Git to local? #2018-05-31 21:53:29 kenny It doesn't seem to work. Example a/deps.edn:
{:deps {org.clojure/tools.cli {:mvn/version "0.3.7"}}}
b/deps.edn:
{:deps {a {:git/url ""
:sha "7b8e5c33588a94c1318e225dc9fe655599216370"}}
:aliases {:dev {:override-deps {a {:local/root "../a"}}}}}
Result of clj -A:dev -Sforce -Spath:
src:/home/kenny/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/home/kenny/.gitlibs/libs/a/a/7b8e5c33588a94c1318e225dc9fe655599216370/src/main/clojure:/home/kenny/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/home/kenny/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar
That classpath does not contain tools.cli as required by the overridden a dependency. #2018-05-31 22:14:06 alexmiller No, there is a ticket for this right now #2018-05-31 22:38:44 alexmiller To allow #2018-06-01 15:29:37 kenny Any way to use aliases defined in a library that is in your :deps? #2018-06-01 15:46:26 alexmiller no #2018-06-01 15:34:22 rickmoynihan FYI :aliases, :resolve-aliases and :classpath-aliases are implemented on 0.4.x branch of lein-tools-deps… just need to kick the tyres a bit and add some documentation before merging to master #2018-06-01 20:10:32 hypirion Is there any way to compile or refer to Java files with deps.edn, or do I have to do that in a separate project? #2018-06-01 20:11:12 dominicm @hypirion Separate #2018-06-01 20:14:03 hypirion Okay, I guess that is by design? It's a bit iffy if I just want to improve the performance of a component in the system by converting it to Java. #2018-06-01 20:31:06 alexmiller in scope: a) make classpaths and b) launch Clojure programs #2018-06-01 20:31:49 alexmiller there are a vast array of other “build” activities you may wish to do #2018-06-01 20:31:59 dominicm I have thought about writing something which scans for .java rules in directories on the classpath and builds them. #2018-06-01 20:32:16 alexmiller well there are about 10 programs that do that already #2018-06-01 20:32:31 dominicm There are? #2018-06-01 20:33:27 alexmiller Sure, Ant, Maven, Leiningen, Boot, Ivy, Grape, #2018-06-01 20:33:31 alexmiller and on and on #2018-06-01 20:34:14 dominicm Sure. They're a bit different. By default they don't expect to compile dependencies, eg local and root dependencies #2018-06-01 20:34:31 alexmiller some yes, some no #2018-06-01 20:34:46 alexmiller you can easily coax Ant or Ivy into this #2018-06-04 08:22:14 borkdude I’m looking into the nix package manager right now and wonder if classpath caching was inspired by this maybe? #2018-06-04 09:40:04 mkvlr @borkdude as someone without fear of parens guix might also be of interest to you. It’s basically nix written in scheme. #2018-06-04 09:41:45 borkdude cool #2018-06-04 10:31:27 robert-stuttaford what are the rules for downloading snapshots? i have two extra :mvn/repos defined (Datomic’s, and our own S3 one), and for some reason, tools.deps is failing on figwheel.main’s dep because it’s not attempting it on the one repo that has it - http://clojars.org. #2018-06-04 10:32:41 robert-stuttaford gosh, nevermind. i had a non-standard space after the dep name, which caused it to be included in the dep name 🙈 #2018-06-04 10:39:52 dominicm that's horribly awesome. #2018-06-04 11:13:32 borkdude maybe write a patch for Clojure to treat that like the other whitespace 😛 #2018-06-04 11:13:56 dominicm How will I drive my co-workers crazy then? #2018-06-04 15:51:22 dominicm I'm having an issue I don't really understand, I'm trying to depend on jnr-posix, with lein all is happy, and with clojure tools the same snippet doesn't work.
com.github.jnr/jnr-posix {:mvn/version "3.0.42"} is the version.
user=> (import
#_=> (jnr.posix POSIXFactory POSIXHandler))
jnr.posix.POSIXHandler
user=> (POSIXFactory/getNativePOSIX)
#object[jnr.posix.LinuxPOSIX 0x6dd5e5ec "
Is the snippet that does work in lein, but not in clojure cli tools.
I even tried doing lein deps :tree and using those dependencies in deps.edn:
{com.github.jnr/jnr-posix {:mvn/version "3.0.42"}
com.github.jnr/jnr-constants {:mvn/version "0.9.9"}
com.github.jnr/jnr-ffi {:mvn/version "2.1.7"}
com.github.jnr/jffi {:mvn/version "1.2.16"}
com.github.jnr/jnr-x86asm {:mvn/version "1.0.2"}
org.ow2.asm/asm-analysis {:mvn/version "5.0.3"}
org.ow2.asm/asm-commons {:mvn/version "5.0.3"}
org.ow2.asm/asm-tree {:mvn/version "5.0.3"}
org.ow2.asm/asm-util {:mvn/version "5.0.3"}
org.ow2.asm/asm {:mvn/version "5.0.3"}}
But that hasn't resolved the issue either. #2018-06-04 15:55:32 dominicm Looks like in the deps.edn version, ( "jni/x86_64-Linux/libjffi-1.2.so") doesn't resolve. #2018-06-04 15:55:52 dominicm OH, I see. It's because jffi has a native classifier. #2018-06-04 15:57:00 dominicm TDEPS-50 and 12. No fun being here 😛 #2018-06-04 15:57:33 dominicm Is there any known workarounds? #2018-06-04 16:10:07 alexmiller you could pull the other classifiers in as local jars #2018-06-04 16:11:38 alexmiller {:deps {madeup/dep {:local/root "path/to/the.jar"}}} #2018-06-04 16:19:02 dominicm @alexmiller this is in pack, I'd like to keep the repo small (no jars in the repo). But I've done that into my maven repo for now, only I need to run the tests, right? #2018-06-04 16:22:16 dominicm Trying to write some tests for pack, which involves changing the jvm directory in order to have :local/root "../blah" work correctly. #2018-06-04 16:52:34 alexmiller I don’t understand what you’re saying, sorry #2018-06-04 17:17:47 dominicm @alexmiller I'm changing the cwd of the process so that relative paths work in deps.edn files. #2018-06-04 17:18:05 dominicm So that I can write tests for pack #2018-06-04 22:31:49 kenny I'd like to make a script to run tests and launch that script with clj. Something like clj -A:test -m test-runner. In order to run tests, you need to know which namespaces you want to test. To determine the namespaces to test you need to find all namespaces given some file system paths. I'd like to be able to figure out the file system paths from the super set of :paths and :extra-paths in your deps config files given the aliases you passed to the clj command. I can't see a way to get the aliases you used to launch the clj command. Any ideas? #2018-06-04 22:39:39 seancorfield @kenny Have you looked at Cognitect Labs' test-runner project? #2018-06-04 22:42:32 kenny Oh cool! Seems like that project takes a different approach by defaulting to test directory otherwise you need to explicitly specify the directory for your tests. I suppose that'd work. #2018-06-04 22:44:59 kenny I still think it would be useful to introspect the clj props used to launch the JVM. #2018-06-04 22:45:31 seancorfield See https://github.com/seancorfield/dot-clojure for an example of using test-runner where the tests could be set by an alias (I specifically run tests with clj -A:test:runner -- where :test can vary by project) #2018-06-04 22:48:55 seancorfield > I can't see a way to get the aliases you used to launch the clj command. Any ideas?
I'm not sure why you'd need/want this? #2018-06-04 22:49:29 seancorfield You can't do anything with those aliases... #2018-06-04 22:49:33 kenny Makes sense. #2018-06-04 22:50:46 kenny There are lots of things I can do with the aliases. I can use them to determine information about the deps.edn file. I can launch sub-processes with the same classpath as the parent process. #2018-06-04 22:52:24 seancorfield So this isn't a real world need, just a "what if"? #2018-06-04 22:52:53 kenny It is real world because that's the exact problem I'm running into right now 🙂 #2018-06-04 22:53:22 seancorfield Can you elaborate? What, specifically, are you trying to do where knowing the actual aliases used would be a solution? #2018-06-04 22:55:19 kenny A simple example would be to get a set of all :paths and :extra-paths used to launch the JVM via clj. #2018-06-04 22:56:11 seancorfield You can already get the classpath in any JVM program... and then you can walk that to see which are directories and which are files... #2018-06-04 22:56:29 kenny Doesn't work with local deps 😉 #2018-06-04 22:56:36 kenny :local/root #2018-06-04 22:57:17 kenny The set of all directories on the classpath may be a super set of :paths, :extra-paths, and other directories with no way to differentiate the three. #2018-06-04 22:58:07 seancorfield And you need to tell them apart because... why? #2018-06-04 22:58:35 kenny Differentiate files in your current project from dependencies. #2018-06-04 22:58:55 kenny I simply want a list of files in :paths and :extra-paths. #2018-06-04 22:59:11 seancorfield Sorry, I'm still not understanding the actual problem you're trying to solve here. I'll just give up and go away. #2018-06-04 23:01:11 kenny Sorry, I should have started with the most precise, minimal problem.
I'd like to get a list of files within the :paths and :extra-paths for the clj launched JVM within the JVM. #2018-06-04 23:02:21 seancorfield Yes, you keep saying that, but I still don't understand what problem you are trying to solve. #2018-06-04 23:07:02 kenny It could be anything - this will be used for general operations over your project files. For example, I may want to run a formatter over all my project files. As done in the test-runner you sent, I could require the user to explicitly pass the directory and default to something like ["src" "test"]. It seems preferable, however, to default to the current set of files within :paths and :extra-paths because these are the files you are currently working with. #2018-06-04 23:13:38 seancorfield Why not use the current working directory where the <insert general tooling> was invoked, and walk the classpath for paths that include that directory as a prefix? Then you have something that is general and could be invoked via clj, lein, or boot. #2018-06-04 23:14:03 seancorfield (with an option to specify paths as -main arguments) #2018-06-04 23:16:03 kenny But why require the user to specify a prefix when they already included the directories they want via aliases? #2018-06-04 23:16:15 seancorfield Re-read what I suggested. #2018-06-04 23:16:26 seancorfield The user would not have to specify a prefix. #2018-06-04 23:17:15 kenny Oh I see. #2018-06-04 23:17:52 seancorfield Then the tooling you produce really is general rather than being hardwired to some implementation artifact of how clj happens to work today. #2018-06-04 23:17:59 kenny I don't believe that is general though. You can specify a path via an alias that is not in the parent path. #2018-06-04 23:19:53 seancorfield If :paths or :extra-paths end up pointing to directories outside the project where you invoked clj (and your imaginary tooling), I think most people would be surprised that it performed operations on those files, rather than what is local inside the current project folder...? #2018-06-04 23:21:34 kenny Perhaps. But they also explicitly included that as a :path instead of a local dependency so they probably have a reason for doing. #2018-06-05 00:36:04 alexmiller this should be considered highly unusual and possibly even a bug #2018-06-04 23:22:55 seancorfield I think you're trying to solve a non-problem and making your tooling set up more complicated (and more brittle) than it needs to be. #2018-06-04 23:23:51 seancorfield Solve the simple, obvious use case first -- using just the current working directory and the classpath, which is the general tool-agnostic case -- and then see if any of your users really do things differently. #2018-06-04 23:24:29 kenny That's fair 🙂 #2018-06-04 23:24:35 seancorfield (I've learned this the hard way with boot-tools-deps -- with hindsight, I would have done that very differently) #2018-06-04 23:25:42 kenny Would you prefer the test-runner project default to directories on the classpath that are prefixed with your current directory or simply the "test" directory? #2018-06-04 23:27:01 seancorfield For a test runner, it makes sense to provide an external, tool-agnostic way to specify where to find tests -- since you already know the source code is going to be on the classpath (as well as the test code). #2018-06-04 23:29:16 seancorfield For example, in simple Clojure projects, src and test are the two top-level folders. In mixed clj/cljs projects, you often have src/clj, src/cljc, src/cljs (and similarly for test) so specifying just what should be run on the JVM is already more complex. And in Maven-style projects, tests are under src/test -- where source code is under src/main (and those tend to have <language> as the next level and then the actual code tree). #2018-06-04 23:33:14 kenny Makes sense. Thanks for the direction. #2018-06-06 04:05:29 cfleming I’m looking into integrating deps support into Cursive. I have some questions about how deps is expected to be invoked for this use case. I’m assuming for the moment that the user must have the command line tools installed, and will have to configure the path to the clojure command. #2018-06-06 04:06:22 cfleming Once I have that, I can then execute with either -Sverbose or -Sdescribe, and get some information about the environment, and I can get a classpath. #2018-06-06 04:06:59 cfleming But I can’t see a way with the command line tool to get path information like source and test paths. #2018-06-06 04:08:41 cfleming I think I’ll have to be using tools.deps.alpha to read e.g. deps.edn files, but I also can’t see a good way to find either the uberjar from the local install, or get the tools.deps.alpha version that I would need to download it myself and use it as a library. #2018-06-06 04:09:40 cfleming Is anything provided for this kind of use case? #2018-06-06 04:10:42 cfleming Also, I’m assuming that deps doesn’t provide any help for downloading source artifacts of dependencies? #2018-06-06 04:19:57 cfleming I just tried clojure -Sdeps '{joda-time/joda-time {:mvn/version "2.9.7" :classifer "sources"}}' -Spath but no luck. #2018-06-06 05:29:59 seancorfield @cfleming Not sure whether you can rely on it "forever" but clojure -A:deps brings in the tools.deps.alpha version... #2018-06-06 05:33:45 seancorfield (and, yes, the tools.deps.alpha library will resolve and download all your dependencies) #2018-06-06 11:28:35 alexmiller On src/test paths - deps.edn doesn’t make this distinction. Normally I’d expects src to be part of the default :paths and test to be provided via an :extra-paths in an alias #2018-06-06 11:29:43 alexmiller On tda version, I could probably add that version to verbose or describe #2018-06-06 11:30:00 cfleming @alexmiller Right, the issue is that I need t.d.a to get that info because I really have to parse deps.edn to get that level of detail. #2018-06-06 11:30:06 cfleming That would be great. #2018-06-06 11:31:34 alexmiller On source artifacts, nothing in clj, but you can easily use stuff in tda directly to do that. Surely IntelliJ already has something though? I guess maybe it relies on a pom? #2018-06-06 11:31:58 cfleming Right, that’s all in the Maven plugin which is an optional dependency. #2018-06-06 11:32:34 cfleming I’ve had problems with users disabling that and then wondering why e.g. autocomplete in project.clj doesn’t work. #2018-06-06 11:33:54 cfleming @alexmiller Is the deps alias going to stick around? #2018-06-06 11:34:23 cfleming If so then I might be able to use clojure -A:deps -Spath, although it’s a little hacky. #2018-06-06 11:36:19 alexmiller Not planning on removing it but I hate to have you rely on that #2018-06-06 11:37:03 cfleming Having some canonical way to build a classpath with the installed version of tda would be nice. #2018-06-06 11:37:43 alexmiller Yeah #2018-06-06 11:37:52 cfleming Even putting the version in -Sdescribe punts to the client and forces it to support Maven resolution. #2018-06-06 11:38:49 cfleming Having the classpath just include the uberjar would be fine too. #2018-06-06 11:39:38 alexmiller The uberjar makes me feel dirty :) #2018-06-06 11:39:47 cfleming Actually, perhaps just add the version and the uberjar path to -Sdescribe? Seems a bit internal… yeah #2018-06-06 11:40:00 cfleming But, you know, it’s already there… #2018-06-06 11:41:01 alexmiller Give me a day or two to think about it #2018-06-06 11:41:16 cfleming Sure #2018-06-06 11:42:51 cfleming @alexmiller Another feature request for the API would be to provide a download progress callback. #2018-06-06 11:43:10 cfleming The Maven stuff supports that, it’s probably trickier for git. #2018-06-06 11:43:25 cfleming Do you use JGit or shell out? #2018-06-06 11:43:35 cfleming JGit might provide something. #2018-06-06 11:44:08 alexmiller Jgit atm. They do have some callback stuff. #2018-06-06 11:44:51 cfleming It would be great to expose that - downloading the deps of something like immutant can take forever. #2018-06-06 11:45:54 alexmiller Expose like via printing or by letting a caller get notification ? #2018-06-06 11:46:27 alexmiller Would be great to get tickets for this kinda stuff #2018-06-06 11:47:45 cfleming I can file some tomorrow. #2018-06-06 11:48:06 cfleming By passing in a function which gets called back. #2018-06-06 11:49:48 cfleming That works nicely. #2018-06-06 11:50:35 cfleming And this: #2018-06-06 11:51:25 cfleming Right, bedtime. I’ll file some tickets tomorrow. #2018-06-06 16:01:44 Hugh Powell Hi folks,
I'm working on a project to get leiningen to source deps.edn for dependency information (lein-tools-deps).
We currently shell out to clojure -Sdescribe to get the vector of :config-files.
This command also returns other information which requires resolution of all a projects dependencies (specifically :resolve-aliases, I think).
Is there any way to get just the :config-files (or all the information that doesn't need full resolution of all dependencies to be completed)? #2018-06-07 08:03:45 Hugh Powell Yep, that's the one #2018-06-07 15:09:23 rickmoynihan @hugh336: Yeah it would be really useful if there was a variant on -Sdescribe that didn’t attempt to resolve your dependencies etc, and only returned the information that is known locally about the installation.
As for us in lein-tools-deps this means that we end up resolving aliases twice once when we don’t really want to #2018-06-07 15:10:34 rickmoynihan I don’t think there’s a good way around this without changing tools deps / or the clj/clojure command line project. #2018-06-07 15:14:48 alexmiller there is actually a flag buried in the underlying clojure program to tell it not to resolve #2018-06-07 15:15:01 alexmiller but it’s not surfaced out to the top #2018-06-07 15:15:50 alexmiller I’d be happy to have a ticket about the problem in TDEPS so I can think about a solution #2018-06-07 15:16:20 rickmoynihan what is the flag? #2018-06-07 15:18:25 alexmiller if set, that avoids doing the resolution, making a classpath, and saving those off #2018-06-07 15:20:00 alexmiller was added for some stuff Stu was doing (in the guts of ion impl) where similar need arose #2018-06-07 15:22:54 alexmiller yes, this flag is at a lower level and not exposed at the script level as I mentioned above #2018-06-07 15:23:45 rickmoynihan cool… we can look at registering an issue #2018-06-07 15:24:37 alexmiller wait, -Sdescribe doesn’t resolve anything #2018-06-07 15:24:53 alexmiller so I will go all the way back to the top of this discussion and disagree with the premise #2018-06-07 15:25:23 rickmoynihan lol ok 🙂 #2018-06-07 15:25:56 alexmiller so this: “Yeah it would be really useful if there was a variant on -Sdescribe that didn’t attempt to resolve your dependencies etc, and only returned the information that is known locally about the installation.” is not correct #2018-06-07 15:26:07 alexmiller that’s exactly what -Sdescribe does #2018-06-07 15:27:07 rickmoynihan ok it’s not what we’re seeing… maybe there’s something else causing us to misdiagnose the problem though #2018-06-07 15:27:23 alexmiller well maybe there’s a bug there #2018-06-07 15:27:53 alexmiller looking at the script, could definitely be but it’s just a matter of another flag check before resolving #2018-06-07 15:28:20 alexmiller so, I’d say that’s the ticket to file - ensure -Sdescribe doesn’t trigger resolution #2018-06-07 15:30:44 Hugh Powell @rickmoynihan @alexmiller Thanks for the discussion, I'll file a ticket #2018-06-07 15:30:55 alexmiller sorry for my densitude #2018-06-07 15:31:49 rickmoynihan not at all, you must be pulled every which way #2018-06-07 15:56:00 rcustodio Hey… I’m using this org.imgscalr/imgscalr-lib {:local/root "libs/imgscalr-lib-4.3.jar"}
But when I do uber with boot, using seancorfield/boot-tools-deps, this jar is not extracted and added to the standalone jar, how do I do it, this is my boot command
BOOT_CLOJURE_VERSION=1.9.0 boot -d seancorfield/boot-tools-deps:0.4.5 \
boot-tools-deps.core/deps -B \
pom -p "zzk/catalog-image-svc" -v "0.0.1" -d "ZZK image processing" -u "" -l "X:X" \
uber \
jar -m clojure.main \
target
#2018-06-07 16:18:01 alexmiller thx #2018-06-07 16:36:33 seancorfield @rcustodio Does it work (run) if you use just clj, not boot? Can you build a JAR with depstar or pack? #2018-06-07 16:36:51 rcustodio Yes, it works with run #2018-06-07 16:36:57 rcustodio clj run #2018-06-07 16:37:11 rcustodio depstar pack, i dunno what that is #2018-06-07 16:37:26 seancorfield Can you put the project up on GitHub so I can clone it and try it out? #2018-06-07 16:37:42 seancorfield depstar and pack are clj-based utilities to create uberjars. #2018-06-07 16:38:06 rcustodio hmm.. I can’t because its a in-company project #2018-06-07 16:38:35 rcustodio That is my deps
{:aliases
{:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.5.435"}}},
:test {:extra-paths ["test"]}},
:deps
{br.com.vikingmakt/augustus {:mvn/version "0.1.12"},
br.com.vikingmakt/njord {:mvn/version "0.3.0"},
cheshire {:mvn/version "5.8.0"},
clj-time {:mvn/version "0.14.4"},
com.novemberain/langohr {:mvn/version "5.0.0"},
com.novemberain/monger {:mvn/version "3.1.0"},
net.mikera/imagez {:mvn/version "0.12.0"
:exclusions [org.imgscalr/imgscalr-lib]},
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.0"},
org.apache.logging.log4j/log4j-slf4j-impl {:mvn/version "2.11.0"},
org.clojure/clojure {:mvn/version "1.9.0"},
org.clojure/tools.logging {:mvn/version "0.4.1"},
org.imgscalr/imgscalr-lib {:local/root "libs/imgscalr-lib-4.3.jar"}},
:mvn/repos
{"central" {:url ""}, "clojars" {:url ""}},
:paths ["resources" "src/main/clojure"]}
#2018-06-07 16:39:13 seancorfield And that imgscalr-lib is your own custom in-house version? #2018-06-07 16:39:35 rcustodio nope, I just got the last (master) from the github repository #2018-06-07 16:39:50 rcustodio Because I’m using something that is not in the latest version from http://maven.org #2018-06-07 16:42:35 rcustodio I will try it out with depster #2018-06-07 16:42:54 seancorfield @rcustodio can you add -vv after -B on the deps task and send me the output via DM here? No point in clogging up the channel with it. #2018-06-07 16:43:04 rcustodio okey #2018-06-07 19:34:19 kenny Not sure if this is a tools-deps problem but if I don't have AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY set as env vars, I get a access denied from a Maven repo. I am specifically trying to access the Datomic cloud repo "" which, I assume, does not require authenticated access. #2018-06-07 21:34:23 alexmiller Yeah, same is under discussion in #datomic - I’m going to dig into it a little more #2018-06-08 03:51:21 cfleming Am I correct in thinking that aliases are not transitive? So if I have a git dep, there’s no way for me to use test code from it? #2018-06-08 03:56:06 cfleming In fact, the same also seems to be true of local deps - is this intentional, or am I missing a way to work around this? #2018-06-08 03:59:29 cfleming I’m also confused by the fact that local deps don’t seem to be transitive. I have a test project, and it has a subproject which the root one depends on using :local/root. The subproject has a git dep, but that dep doesn’t appear on the classpath for the root project. #2018-06-08 04:35:12 ghadi aliases being non transitive is intentional #2018-06-08 04:36:39 ghadi As for the other comment, local deps should be transitive (provided a pom.xml or deps.edn) #2018-06-09 02:51:03 cfleming @ghadi Local deps are definitely not transitive at the moment - I haven’t tested git deps. #2018-06-09 02:51:51 ghadi Bug #2018-06-09 03:02:41 cfleming Actually, I might be going nuts - I can’t reproduce what I was seeing yesterday. #2018-06-09 03:04:50 cfleming I wonder if this was a caching problem: #2018-06-09 03:05:11 cfleming ~/d/t/test-project (master)> tree .
.
├── deps.edn
└── sub-project
├── deps.edn
└── sub-sub-project
└── deps.edn
2 directories, 3 files
~/d/t/test-project (master)> cat deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "13e9d244678be7b235bb24a10310f9d147ea088d"}
test/sub-project {:local/root "sub-project"}}}
~/d/t/test-project (master)> cat sub-project/deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "0f06ca5eb08726e74b2dab8654923894b7ace6af"}
test/sub-sub-project {:local/root "sub-sub-project"}}}
~/d/t/test-project (master)> cat sub-project/sub-sub-project/deps.edn
{:deps {}}
~/d/t/test-project (master)> clojure -Spath
Error building classpath. Manifest type not detected when finding deps for test/sub-sub-project in coordinate #:local{:root "sub-sub-project"}
~/d/t/test-project (master)> clojure -Spath -C:test
test:src:/Users/colin/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/Users/colin/.gitlibs/libs/org.clojure/data.json/13e9d244678be7b235bb24a10310f9d147ea088d/src/main/clojure:/Users/colin/dev/tools.deps.alpha/test-project/sub-project/src:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar
#2018-06-09 03:05:49 cfleming Here the first classpath with no alias gives an error that the manifest is not supplied. #2018-06-09 03:06:06 cfleming The second doesn’t give that error, but returns an incorrect classpath. #2018-06-09 03:06:40 cfleming I add the manifests: #2018-06-09 03:06:45 cfleming ~/d/t/test-project (master)> tree .
.
├── deps.edn
└── sub-project
├── deps.edn
└── sub-sub-project
└── deps.edn
2 directories, 3 files
~/d/t/test-project (master)> cat deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "13e9d244678be7b235bb24a10310f9d147ea088d"}
test/sub-project {:local/root "sub-project"
:deps/manifest :deps}}}
~/d/t/test-project (master)> cat sub-project/deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "0f06ca5eb08726e74b2dab8654923894b7ace6af"}
test/sub-sub-project {:local/root "sub-sub-project"
:deps/manifest :deps}}}
~/d/t/test-project (master)> cat sub-project/sub-sub-project/deps.edn
{:deps {}}
#2018-06-09 03:07:15 cfleming And then both work: #2018-06-09 03:07:19 cfleming ~/d/t/test-project (master)> clojure -Spath
src:/Users/colin/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/Users/colin/.gitlibs/libs/org.clojure/data.json/13e9d244678be7b235bb24a10310f9d147ea088d/src/main/clojure:/Users/colin/dev/tools.deps.alpha/test-project/sub-project/src:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar:/Users/colin/dev/tools.deps.alpha/test-project/sub-sub-project/src
~/d/t/test-project (master)> clojure -Spath -C:test
test:src:/Users/colin/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/Users/colin/.gitlibs/libs/org.clojure/data.json/13e9d244678be7b235bb24a10310f9d147ea088d/src/main/clojure:/Users/colin/dev/tools.deps.alpha/test-project/sub-project/src:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar:/Users/colin/dev/tools.deps.alpha/test-project/sub-sub-project/src
#2018-06-09 03:08:26 cfleming Then if I remove the manifest, I get the error with both: #2018-06-09 03:08:31 cfleming ~/d/t/test-project (master)> cat deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "13e9d244678be7b235bb24a10310f9d147ea088d"}
test/sub-project {:local/root "sub-project"}}}
~/d/t/test-project (master)> cat sub-project/deps.edn
{:deps {org.clojure/data.json {:git/url ""
:sha "0f06ca5eb08726e74b2dab8654923894b7ace6af"}
test/sub-sub-project {:local/root "sub-sub-project"}}}
~/d/t/test-project (master)> clojure -Spath
Error building classpath. Manifest type not detected when finding deps for test/sub-sub-project in coordinate #:local{:root "sub-sub-project"}
~/d/t/test-project (master)> clojure -Spath -C:test
Error building classpath. Manifest type not detected when finding deps for test/sub-sub-project in coordinate #:local{:root "sub-sub-project"}
#2018-06-09 04:08:35 alexmiller too tired to read all that atm but there is a known issue that the caching will not notice when local dep project classpaths change, so you need to -Sforce to recompute. not sure if that’s what you’re seeing. I have a plan to fix that just haven’t had time to work on it yet. #2018-06-09 04:09:39 alexmiller if you think there’s a bug, please feel free to drop it in a ticket so I can look at it later as this will get lost #2018-06-09 19:49:59 gfredericks what's the right way to call clojure with a script going to stdin?
it works okay with no args, but prints a bunch of repl prompts I don't want #2018-06-09 21:24:52 alexmiller We don’t have that level of repl control with clojure.main #2018-06-09 21:25:31 alexmiller Well, maybe the repls not involved there? #2018-06-09 21:27:48 gfredericks $ echo '(println 42)' | clojure
Clojure 1.9.0
user=> 42
nil
user=>
is what I'm referring to #2018-06-09 21:28:54 alexmiller Isn’t there a - option you can pass? #2018-06-09 21:29:18 gfredericks there is, that's probably all I need #2018-06-09 21:42:11 richiardiandrea Oh awesome didn't know that either 😃 #2018-06-10 15:39:41 borkdude Cool, I needed this once too #2018-06-10 16:07:31 ghadi I do that all the time. I like datomic ion's take on it too. #2018-06-10 16:12:48 ghadi Sending in an edn map through an arg or stdin #2018-06-10 17:00:29 gfredericks ironically I quickly realized that I couldn't do that in this case, because the program also took its input from stdin #2018-06-10 17:01:29 gfredericks I guess in theory I could have arranged for the script to be prepended to the input and make sure the last form is one that does the work and then System/exits, but that sounds pretty hacky #2018-06-10 17:05:19 borkdude @gfredericks I once asked for an option in here to provide a program with the -e option, without printing expressions. #2018-06-10 17:05:26 borkdude Would that make sense in your case? #2018-06-10 17:05:47 gfredericks I could've but it was a ~20 line script so I felt weird putting that in an arg #2018-06-10 17:06:03 borkdude put it in a variable. #2018-06-10 17:06:18 dominicm clojure -e <<EOF
(println "ta-da")
EOF
#2018-06-10 17:06:25 dominicm heredoc can be pretty useful #2018-06-10 17:07:05 dominicm (apparently that doesn't work) #2018-06-10 17:07:25 borkdude {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "f8ba978a939c8d9c9db89a978a939c8d9c9d"}, :content ("[email protected]")} #2018-06-10 17:08:48 borkdude But then again, you could just do clj hello.clj 🙂 #2018-06-10 17:13:43 gfredericks I'm not a bash lawyer yet, but I got the distinct impression yesterday that <<EOF goes to stdin #2018-06-10 17:23:31 dominicm clojure -e "$(cat << 'FOO'
(println "ta-da")
FOO
)"
You're right, I've just got on a journey learning that 🙂 #2018-06-10 17:24:28 gfredericks I've been learning some bash recently.
Weirdest thing so far is the <> redirection operator. #2018-06-10 17:26:49 dominicm what does that do? #2018-06-10 20:08:12 gfredericks same as < except it makes stdin writable #2018-06-11 12:02:57 delaguardo Is it possible to specify init-opt within deps.edn file? #2018-06-11 12:19:18 alexmiller No, not right now #2018-06-11 17:11:09 bhauman should I be surprised that clojure forms in main opts are not output as quoted args? #2018-06-11 17:11:51 bhauman I.E. :main-opts ["-e" (list 1 2 3)] #2018-06-11 17:12:24 bhauman the list expression gets broken up into separate args #2018-06-11 17:13:17 bhauman I guess escaping gets complicated #2018-06-11 17:14:30 bhauman or there is a depth of features here I don't understand #2018-06-11 17:21:02 hiredman I think the thing you might be missing is :main-opts is meant to be a list of strings passed as command line arguments, not a list of arbitrary datastructures #2018-06-11 17:25:39 seancorfield Also remember that , is whitespace so you can often get around shell-level tokenization by using , in strings instead of spaces. But, yeah, :main-opts are strings @bhauman. #2018-06-11 17:26:42 bhauman thanks guys #2018-06-11 17:27:19 bhauman another question, is this not a feature that we might want? #2018-06-11 17:28:27 bhauman to take advantage of the fact that we are in a clojure form to begin with and script our way to euphoria? #2018-06-11 17:29:36 seancorfield But :main-opts are what is passed as a sequence of strings to the -main function via clojure.main -- it's all strings. You can easily pass an EDN string if you want a data structure. #2018-06-11 17:29:50 seancorfield There's no difference between command-line invocation and :main-opts. #2018-06-11 17:30:20 seancorfield and it is passed by a script, not by Clojure code. #2018-06-11 17:31:06 bhauman the script doesn't parse the deps.edn #2018-06-11 17:31:15 seancorfield The :main-opts are cached to a text file and then piped into the (string) options passed to clojure.main. Just like the :jvm-opts. #2018-06-11 17:31:16 bhauman does it? #2018-06-11 17:31:51 bhauman I get this but I'm sure clojure code prepares the opts #2018-06-11 17:31:59 bhauman before caching #2018-06-11 17:33:07 seancorfield It's just strings in the cached file. And it's just strings that are passed to clojure.main. #2018-06-11 17:34:34 bhauman I must be missing something, are you saying that the feature isn't possible? #2018-06-11 17:35:01 bhauman or that it messes up the semantics? #2018-06-11 17:35:26 seancorfield Take a look at the clojure shell script. Here's an example of what gets cached: (! 763)-> ls -l .cpcache/*.main
-rw-r--r-- 1 sean staff 90 Mar 26 15:38 .cpcache/1770942858.main
-rw-r--r-- 1 sean staff 17 Apr 14 00:52 .cpcache/2142975460.main
-rw-r--r-- 1 sean staff 17 Apr 14 18:18 .cpcache/4269137912.main
-rw-r--r-- 1 sean staff 22 May 2 14:38 .cpcache/84686696.main
(sean)-(jobs:0)-(~/clojure)
(! 764)-> cat .cpcache/84686696.main
-m rebel-readline.main
(sean)-(jobs:0)-(~/clojure)
(! 765)-> cat .cpcache/1770942858.main
-e (require,'[clojure.tools.nrepl.server,:refer,[start-server]]),(start-server,:port,5555)
(sean)-(jobs:0)-(~/clojure)
(! 766)->
#2018-06-11 17:37:48 bhauman yes but we call the tools.aplha to resolve the tags #2018-06-11 17:38:20 bhauman which is parsing main opts and therefore could quote clojure forms before caching them? #2018-06-11 19:07:44 hagmonk I just made a fairly big update to https://github.com/hagmonk/depify/ … would appreciate any folks here who could kick the tires a bit before I post to the mailing list #2018-06-11 19:14:23 seancorfield This includes adding some test runner boilerplate from clj-new. -- not sure what you mean there @hagmonk? #2018-06-11 19:15:30 seancorfield Should that link to Cognitect's test-runner repo perhaps? #2018-06-11 19:17:40 hagmonk Oh I pinched that from a clj-new generated app - specifically the separate test and runner aliases #2018-06-11 19:18:40 hagmonk I didn’t see the same style being used on the test-runner repo, but maybe I missed it #2018-06-11 19:19:44 hagmonk Cool! I will make that change #2018-06-11 19:22:36 hagmonk I’m exploring the idea of adding more aliases to provide parity with project.clj, in those cases where someone has a tools.deps based tool we can use #2018-06-11 19:22:48 hagmonk CLJS and uberjars for instance #2018-06-11 19:25:43 hagmonk so now I'm looking at your deps.edn file and getting lots of ideas :) but I need to work on my real projects today at some point, LOL #2018-06-11 20:21:15 alexmiller @bhauman I didn’t understand your earlier comments re quoting. if it’s something I should pay attention to, please try again :) #2018-06-11 20:21:35 bhauman happy to 🙂 #2018-06-11 20:21:53 alexmiller the ground truth is that the jvm takes a classpath, a main class and string args to the main class. so we want to expose that, first. #2018-06-11 20:22:08 alexmiller other things on top of that may also be useful #2018-06-11 20:22:14 bhauman absolutely #2018-06-11 20:22:36 alexmiller but I didn’t understand what problem you were trying to solve or thing you were trying to make better or avoid #2018-06-11 20:23:36 bhauman it just seems there is an opportunity when providing clojure forms (not strings) inline in :main-opts to output a single string arg that will make it through to the jvm process as a single arg #2018-06-11 20:25:58 alexmiller the ultimate call needs individual string args though - how do you decide how to parse and tokenize? #2018-06-11 20:28:21 bhauman :main-opts ["-e" {:build 1}] => clojure -e "{:build 1}" #2018-06-11 20:28:36 bhauman is what I'm imagining #2018-06-11 20:29:39 alexmiller you seem to be suggesting a read/print cycle but using different kinds of printing #2018-06-11 20:30:34 alexmiller “-e” would be read as a string and pr’ed as “-e” (perhaps escaped in the invocation as clojure -e '"-e"` #2018-06-11 20:32:16 bhauman it seems that string typs can go though untouched and that forms would be dispatched as such #2018-06-11 20:32:27 bhauman other than string forms 🙂 #2018-06-11 20:32:53 bhauman its a common need, and it seems like a shame to be in a clojure form and then have to translate and escape a clojrue form correctly #2018-06-11 20:33:31 alexmiller there are a couple layers of transformation here and I intend to sit down and sort through them methodically. I’m well aware it’s broken in various ways at the moment. #2018-06-11 20:33:57 alexmiller I’m a little concerned about implying evaluation in the deps.edn file #2018-06-11 20:34:27 alexmiller the args there are read (as edn) and never evaluated and I don’t want to imply that (or do it) #2018-06-11 20:34:48 alexmiller it’s important to me that it remains plain edn data #2018-06-11 20:35:11 bhauman absolutely #2018-06-11 20:36:03 bhauman not what I'm thinking or wanting to imply at all #2018-06-11 20:36:22 alexmiller I didn’t think so, but it’s easy to get that impression from that example #2018-06-11 20:36:55 alexmiller I’m not a fan of “special” arg handling #2018-06-11 20:36:57 bhauman where I really want it is for cljs.main and figwheel.main to supply inline config #2018-06-11 20:37:25 alexmiller yeah, that’s really same need as say, server repl #2018-06-11 20:37:49 alexmiller passing arbitrary edn as an arg #2018-06-11 20:38:04 bhauman :main-opts ["-m" "cljs.main" "-co" {:output-to "out/main.js"} ] #2018-06-11 20:38:07 bhauman yeah #2018-06-11 20:38:19 alexmiller in a way that is not a nightmare, and that works :) #2018-06-11 20:38:35 bhauman what works? #2018-06-11 20:38:47 alexmiller I’m just saying that a requirement is that the end result works #2018-06-11 20:38:58 alexmiller (as it’s broken now) #2018-06-11 20:39:03 bhauman gotcha 🙂 #2018-06-11 20:40:16 bhauman it is interesting though that I'd expected this to work #2018-06-11 20:40:37 bhauman I intuited that it might me a likely feature #2018-06-11 20:40:59 alexmiller there might be a higher level solution to this problem too - I know Rich has mentioned various ideas that are in this ballpark but a somewhat different take on it #2018-06-11 20:41:47 alexmiller for example, you could expose functions directly and hide the main goo entirely #2018-06-11 20:42:06 bhauman that is interesting #2018-06-11 20:42:28 alexmiller it’s come up in several contexts and I know he’s noodling about it #2018-06-11 20:42:52 alexmiller but sometimes he noodles for many years before it lands somewhere :) #2018-06-11 20:43:23 alexmiller so I make no suggestion that a deux ex machina will save us all here #2018-06-11 20:44:07 alexmiller clojure.main itself is both very flexible and frustratingly limited #2018-06-11 20:44:38 alexmiller like I think it sucks that rebel-readline couldn’t just be hooked directly #2018-06-11 20:44:57 bhauman I had some thoughts about that #2018-06-11 20:45:43 bhauman I could potentially graalvm it, and provide a rlwrap equivalent #2018-06-11 20:45:44 alexmiller well, I need to roll back to other work, but I would like to hear them somewhere :) #2018-06-11 20:46:05 bhauman thanks Alex! later #2018-06-11 20:46:15 alexmiller later… #2018-06-12 10:54:26 borkdude hmm, native rebel-readline… interesting #2018-06-12 20:58:09 hagmonk Is there some escaping trick for supplying edn inside tools.deps aliases? #2018-06-12 20:58:19 hagmonk :cljs {:main-opts ["-m" "cljs.main" "--compile-opts" "{:asset-path \"out\" :output-dir \"resources/public/out\"}" "--compile" "access-manager.client" ]}}
#2018-06-12 20:58:54 mfikes Use a lot of non-whitespace whitespace (aka , #2018-06-12 20:59:07 bhauman replace the whitespace with commas #2018-06-12 23:50:11 richiardiandrea wouldn't go around telling colleagues about this though, we might be mistaken for a bunch of JS devs 😄 😄 🙂 #2018-06-12 23:56:03 zentrope With deps.edn + cljs.main + figwheel.main, the tooling is so simple and streamlined that I just can't see how JS devs put up with that they do. #2018-06-13 00:01:51 hagmonk where's the cutting edge example for figwheel.main? I was just about to go looking for that #2018-06-13 00:02:55 bhauman @hagmonk working on a template right now #2018-06-13 00:03:35 bhauman figwheel-main will be a tough conversion though #2018-06-13 00:03:46 bhauman config names are different #2018-06-13 00:04:40 bhauman but actually it should be pretty straightforward for most projects #2018-06-13 00:07:23 hagmonk I'm sure there are some leiningen-isms that are not possible to bring across, or considered an anathema in the eyes of deps.edn, but I'd like to bootstrap people to the greatest extent possible #2018-06-13 00:07:46 hagmonk I'm already generating Makefiles for my internal projects, so that particular pandora's box is open #2018-06-13 00:19:11 bhauman Yeah just created a Makefile myself, I forgot how dang effective they are #2018-06-13 00:19:59 hagmonk Right?? Quick and painless … it had been at least 10 years since I wrote one #2018-06-13 00:20:31 bhauman yeah and it can get pretty sophisticated with relatively little syntax #2018-06-13 00:20:55 bhauman its like it was built for this particular job #2018-06-13 00:21:13 hagmonk Just remember that literal $ signs need to be $$ to stop make from interpreting them :) Got caught by that one #2018-06-13 00:22:20 bhauman I have noticed something interesting, if you use lein without plugins and only use the profiles and aliases, the performance is pretty similar #2018-06-13 00:22:49 bhauman or significantly improved #2018-06-13 00:23:38 hagmonk fascinating … perhaps plugin dependencies are in serial after everything else #2018-06-13 00:23:52 bhauman they are dynamicly loaded #2018-06-13 00:24:00 bhauman which is what causes the hit #2018-06-13 00:27:04 hagmonk inherent to changing the classpath dynamically, or something specific with clojure like having to compile? #2018-06-13 00:27:47 bhauman clojure needing to compile #2018-06-13 00:30:36 hagmonk If the jar is in the classpath beforehand, is compilation skipped or just faster because the JVM gets to batch it all up at once? #2018-06-13 00:32:45 bhauman lein is aot compiled #2018-06-13 00:33:11 bhauman so... fast by itself #2018-06-13 00:33:50 zentrope If someone made a super quick class-path-maveny thing in non-Java, make would sing. #2018-06-13 01:17:26 alexmiller that would be very tedious #2018-06-13 01:17:39 alexmiller however, you could graalvm the classpath maker #2018-06-13 02:59:43 hagmonk hmm, @bhauman - java.lang.RuntimeException: No such var: cljs.repl/*repl-env* #2018-06-13 03:00:52 bhauman @hagmonk that would happen if you were somehow getting an earlier version of CLJS than 1.10 #2018-06-13 03:01:09 bhauman we can talk about this in #figwheel-main #2018-06-13 03:01:55 hagmonk ah, didn't know that existed :) #2018-06-13 17:15:46 bhauman tools-deps does a silent failure when an alias doesn't exist #2018-06-13 17:17:58 dominicm s/silent failure/ignores #2018-06-13 17:37:11 alexmiller ticket welcome #2018-06-13 17:37:36 alexmiller I thought I actually had a check for that, but maybe not #2018-06-13 17:39:12 dominicm I don't know it's strictly a problem tbh #2018-06-13 17:47:32 alexmiller seems like it’s more likely to be a typo than cleverness #2018-06-13 19:10:11 hagmonk 13th law of software development: itches are to be scratched by the one who itches. #2018-06-13 19:10:49 alexmiller I’ve used the lein stuff for this a bit and it’s SUPER slow #2018-06-13 19:11:06 alexmiller so slow that I just go use the equivalent web pages instead #2018-06-13 19:11:12 alexmiller I have no idea why #2018-06-13 19:11:20 alexmiller but maybe worth understanding #2018-06-13 19:12:19 alexmiller some next level stuff would be to think about bridging the gap between ns and dep #2018-06-13 19:12:35 dominicm @alexmiller what are the important features of the search? I've got a week of open source and JUXT are interested in deps.edn. #2018-06-13 19:13:03 alexmiller lein search does approximately what I want, I just find it too slow to use #2018-06-13 19:13:32 alexmiller I want google for deps #2018-06-13 19:13:39 dominicm Startup or runtime? It will almost certainly be called find, because 4 letters. #2018-06-13 19:13:59 alexmiller runtime is the part I’m referring to #2018-06-13 19:14:22 dominicm But mavens is fast? #2018-06-13 19:14:25 alexmiller I think there might be some giant index file that gets downloaded periodically? #2018-06-13 19:14:33 alexmiller I don’t remember what maven has for this #2018-06-13 19:14:42 alexmiller maybe I’m wrong that they do #2018-06-13 19:15:16 alexmiller I just tried lein search again and it was pretty fast #2018-06-13 19:15:35 alexmiller been a while #2018-06-13 19:16:19 alexmiller how about this - I’d like to do lein search functionality without being tied to lein #2018-06-13 19:16:40 dominicm Or tools.deps.alpha! #2018-06-13 19:16:45 dominicm Just a main #2018-06-13 19:17:55 alexmiller no reason it needs to be there #2018-06-13 19:18:13 dominicm I agree. #2018-06-13 19:18:24 alexmiller I don’t think maven actually has apis for this - it’s site-specific functionality iirc #2018-06-13 19:18:32 alexmiller (which also means they are different :) #2018-06-13 19:18:55 dominicm Clojars has an index which mentions nexus. #2018-06-13 19:19:22 alexmiller well, maybe they’re the same then, not sure #2018-06-13 19:19:39 dominicm Maybe. No idea. Just a data point. #2018-06-13 19:19:53 alexmiller the next level and far more interesting problem is how to connect namespaces and artifacts #2018-06-13 19:20:17 dominicm I'm super interested in this. #2018-06-13 19:20:27 dominicm Solving this makes writing plugins easier. #2018-06-13 19:20:47 dominicm Both for different kinds of artefact, and for build tools. #2018-06-13 19:20:55 alexmiller Rich and I have talked about it a lot but it’s a place where Maven doesn’t really help you much #2018-06-13 19:23:33 dominicm What would a straw man solution look like? I'm struggling to picture something. #2018-06-13 19:30:33 alexmiller I don’t have one :) #2018-06-13 19:30:56 alexmiller sorry, other things I need to focus on atm #2018-06-13 22:32:05 danielcompton Lein now uses the Clojars api iirc. It used to download some kind of massive index which was very slow #2018-06-14 00:03:30 alexmiller Glad I’m not imagining that, and that it doesn’t now! It did seem plenty fast #2018-06-13 23:20:48 bhauman @seancorfield trying to use clj-new with a locally installed lein template and not getting very far #2018-06-13 23:32:35 seancorfield @bhauman Can you be a bit more specific? #2018-06-13 23:32:41 seancorfield "locally installed"? #2018-06-13 23:33:05 bhauman I did a lein install on a template I'm working on locally #2018-06-13 23:34:00 bhauman should it be able to resolve it? #2018-06-13 23:34:15 bhauman as it dies in clj_new.helpers$resolve_remote_template.invokeStatic(helpers.clj:64) #2018-06-13 23:34:53 seancorfield Hmm, my normal process for testing templates is to install them locally and then try to use them via new... #2018-06-13 23:35:13 bhauman here is the message #2018-06-13 23:35:14 bhauman Exception in thread "main" clojure.lang.ExceptionInfo: Unable to locate Clojure's edn files {:exit 1, :out "Invalid option: -Sdescribe\n", :err ""} #2018-06-13 23:35:51 seancorfield Ah, that's where it shells out (via tools.deps.alpha) to run clojure -Sdescribe... #2018-06-13 23:36:08 seancorfield So the error is coming from tools.deps.alpha not being able to run that command. #2018-06-13 23:36:38 bhauman and so there needs to be a deps.edn in the current dir? #2018-06-13 23:36:46 seancorfield If you type clojure -Sdescribe in the folder where you are running clj-new what do you get? #2018-06-13 23:36:57 seancorfield No, this isn't about deps.edn #2018-06-13 23:37:14 bhauman invalid option #2018-06-13 23:37:17 bhauman I have to update #2018-06-13 23:37:36 seancorfield How old is your version of clojure? #2018-06-13 23:37:54 seancorfield Here's what I see in a folder without deps.edn: (! 665)-> clojure -Sdescribe
{:version "1.9.0.375"
:config-files ["/usr/local/Cellar/clojure/1.9.0.375/deps.edn" "/Users/sean/.clojure/deps.edn" ]
:install-dir "/usr/local/Cellar/clojure/1.9.0.375"
:config-dir "/Users/sean/.clojure"
:cache-dir "/Users/sean/.clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
#2018-06-13 23:38:27 seancorfield I thought -Sdescribe had been in there for ages... 👀 #2018-06-13 23:38:37 bhauman or not #2018-06-13 23:39:54 bhauman yeah its not in my help string #2018-06-13 23:40:00 bhauman so upgrading #2018-06-13 23:41:13 bhauman @seancorfield and you are enforcing that project names are multisegment and qualified? #2018-06-13 23:41:23 bhauman shouldn't that be up to the template? #2018-06-13 23:42:20 seancorfield By popular demand, I am disallowing single-segment project names, yes. You can either use a multi-segment name or a qualified name (or both). #2018-06-13 23:42:44 seancorfield Some templates don't accept a qualified name -- which I consider a bug in those templates 🙂 #2018-06-13 23:42:54 bhauman @seancorfield it worked 🙂 but I'd like to be able to specify a single segement name like lein allows #2018-06-13 23:43:07 seancorfield If they don't accept a qualified name, they should accept a multi-segment name. #2018-06-13 23:43:53 bhauman the lein app template morphs a single segment name into a name.core #2018-06-13 23:44:09 seancorfield Yes, which is a terrible idea. #2018-06-13 23:44:22 seancorfield Even Phil says so. #2018-06-13 23:44:38 seancorfield You can always do this: (! 666)-> clj -A:new app bhauman.core
Generating a project called bhauman.core based on the 'app' template.
Wed Jun 13 16:43:52
(sean)-(jobs:0)-(~/clojure/scratch)
(! 667)-> tree bhauman.core/
|____
| |____.gitignore
| |____.hgignore
| |____CHANGELOG.md
| |____deps.edn
| |____doc
| | |____intro.md
| |____LICENSE
| |____README.md
| |____resources
| | |____.keep
| |____src
| | |____bhauman
| | | |____core.clj
| |____test
| | |____bhauman
| | | |____core_test.clj
#2018-06-13 23:46:32 gfredericks is (! 666)-> your bash prompt #2018-06-13 23:47:13 gfredericks er; I guess the last of three lines of it #2018-06-13 23:47:45 seancorfield Ah, yes, my crazy bash prompt... just a sec, I'll get the code for that... #2018-06-13 23:48:01 seancorfield #
PS1="\n\d \t\n\[\e[32;1m\](\[\e[37;1m\]\u\[\e[32;1m\])-(\[\e[37;1m\]jobs:\j\[\e[32;1m\])-(\[\e[37;1m\]\w\[\e[32;1m\])\n(\[\[\e[37;1m\]! \!\[\e[32;1m\])-> \[\e[0m\]"
#2018-06-13 23:48:02 bhauman @seancorfield OK well I don't see the disturbing problem with using a single segment when you are developing a webapp #2018-06-13 23:48:30 bhauman thats going to be deployed to heroky #2018-06-13 23:48:56 bhauman I should call it heroky from now on #2018-06-13 23:49:00 seancorfield Then use <whatever>.core as your project name and you'll get what Leiningen does. #2018-06-13 23:49:53 bhauman thanks I understand that, was just wanting to keep things familiar for folks who are used to doing lein new figwheel hello-world #2018-06-13 23:50:35 bhauman but not a big deal #2018-06-13 23:50:57 bhauman clj -Anew figwheel-main hello.core works #2018-06-13 23:53:49 seancorfield See comments about ClojureScript in that thread, as well as Alex's comment. #2018-06-13 23:54:10 bhauman you do realize I'm not advocating single segment namespaces? #2018-06-13 23:54:33 seancorfield You seemed to be when you complained that clj-new disallowed them 🙂 #2018-06-13 23:55:34 bhauman a single segment argument atumatically gets a "core" segment appended #2018-06-13 23:55:40 seancorfield I haven't -- yet -- disallowed them in boot-new but I may well propose that. #2018-06-13 23:56:11 bhauman single segments don't work I get that #2018-06-13 23:56:44 seancorfield Yes, and having thousands of projects with spurious .core namespaces, just because we were all too lazy to teach beginners to name things properly, is not a great situation to be in. #2018-06-13 23:59:35 bhauman OK so I'm not asking for single-segment namespaces, and defaulting to core is a bad idea for an initial template #2018-06-14 00:00:52 bhauman I'm also betting that defaulting to "app" rather than core doesn't work for you either #2018-06-14 00:02:35 seancorfield People should be explicit about their project namespace. #2018-06-14 00:05:09 bhauman and my point of view is that the name in the template can be more general than a project name, and it can be helpful to let folks who write templates do what they want. #2018-06-14 00:06:33 bhauman the clj generators come in handy there #2018-06-14 03:11:37 hagmonk Just a heads up, I pushed a new https://github.com/hagmonk/depify that now includes the ability to read project.clj from stdin, as well as interpreting output from lein pprint #2018-06-14 20:57:35 hagmonk Hey so riddle me this ... has any thought been given to using deps.edn in the context of a monorepo? #2018-06-14 20:58:04 hagmonk I’m not sure it’s currently possible to supply a path to a directory inside a larger git repo - perhaps I just missed that feature? #2018-06-14 21:04:50 seancorfield We're working with a monorepo at work and slowly figuring that out. Not a priority tho' so progress is a bit slow. #2018-06-14 21:06:35 alexmiller as a git dep or a local dep? #2018-06-14 21:07:01 alexmiller there are some ways with :deps/root etc to do some of this but also some known issues #2018-06-14 21:10:26 seancorfield I'm working with local deps so far. #2018-06-14 21:14:38 hagmonk I guess in the context of the monorepo itself, you would use :deps/root to point at other resources in the monorepo ... but clients outside the monorepo have no way to point at a subfolder’s deps.edn #2018-06-14 21:15:54 hagmonk AFAICT, there’s no way I can depend on that deps.edn file, because it’s not at the root of the repo #2018-06-14 21:28:23 kenny @hagmonk Setting :deps/root to "figwheel-main" doesn't work? #2018-06-14 21:28:59 hagmonk Oh, perhaps that is the thing I’m missing! #2018-06-14 21:29:18 kenny {:git/url ""
:sha "6368c0e49108fc7a8ec99df05831ff289123f1a5"
:deps/root "figwheel-main"}
🙂 #2018-06-14 21:29:45 hagmonk Thanks @kenny! Had no idea that existed #2018-06-14 21:30:15 kenny Yeah I had the same problem. I don't think it's documented yet. #2018-06-14 22:23:02 alexmiller it’s not #2018-06-14 22:24:05 bhauman good to know! #2018-06-15 15:38:24 richiardiandrea @michal interesting work you have shared there thank you! #2018-06-15 15:46:39 michal @richiardiandrea you're welcome. this is something I had on my mind since I saw JUXT's edge project first time. I hope it will be useful at least for some people here 🙂 #2018-06-16 01:16:57 hagmonk @richiardiandrea @michal what are you guys referring to? I think it might have scrolled out of the channel history … #2018-06-16 01:26:21 hagmonk That's cool! #2018-06-16 01:27:20 hagmonk how did I not know about Capsule before this … #2018-06-16 04:06:30 dominicm It took me so long to find #2018-06-16 04:06:37 dominicm And I was really looking #2018-06-17 00:31:24 hagmonk Alright @alexmiller, I've almost finished that "google for deps without lein" tool you were wanting :) Just putting the finishing touches on the readme … #2018-06-17 01:09:01 hagmonk I think this is one tool I actually started using while I was in the middle of writing it. Didn't realize how badly I wanted it :) No more going to google to find dep coords #2018-06-17 02:30:54 mfikes ^ Very nice 🙂 #2018-06-17 21:09:33 hagmonk Ok, who here knows something about stdin/stdout flushing? I'm adding the ability to pipe find-deps commands together, and having a devil of a time getting that to work #2018-06-17 21:38:50 hagmonk for posterity:
1) shopt -u lastpipe is evil, and I had to disable that (`shopt -s lastpipe`)
2) (print "") (flush) at the top of -main will prime the stdout stream such that other processes connected via pipes can test for this condition with (pos? (.available System/in)) #2018-06-17 22:44:32 hagmonk Latest find-deps was just pushed, it now has the ability to write updates to your deps.edn file. I posted an announcement to the Clojure list too #2018-06-17 22:45:04 hagmonk E.g. adding metosin/jsonista is now as easy as: clj -A:find-deps -F:save jsonista #2018-06-17 22:45:38 hagmonk I haven't really found any cases yet where the fuzzy matching has picked totally the wrong lib - that's probably the number one thing you can bring to my attention if you see it fail #2018-06-18 01:41:48 cfleming @hagmonk That looks lovely! #2018-06-18 01:42:31 cfleming Searching is by artifact/group name, right? AFAIK you can’t use the Maven APIs to search for namespace names? #2018-06-18 01:43:14 hagmonk yes, just the group/artifact string #2018-06-18 01:44:29 cfleming We really need something like Maven’s search by classname for namespaces and vars, but the indexing required for that is non-trivial. #2018-06-18 01:45:23 hagmonk right, I did some poking around the crossclj sources, seems like the fellow maintaining that has taken a step back from it #2018-06-18 01:45:34 cfleming Yes, he has #2018-06-18 01:46:00 cfleming I’m not sure he maintained it much past the initial implementation, but I might be wrong there. #2018-06-18 01:46:21 hagmonk It would be great to have a CLJ/CLJS source code index that also tracked the source repo and any built artifacts spun out of that #2018-06-18 01:47:38 cfleming I have a question about deps support for Cursive. As I envision it, each deps project (i.e. a deps.edn, basically) would be a module in IntelliJ, so modules will be created for local deps etc, and libraries created for git deps. #2018-06-18 01:48:15 cfleming My problem is that modules have to be given a name, and it has to be something that I can uniquely determine from the deps.edn file, which AFAIK doesn’t provide anything like that. #2018-06-18 01:48:41 cfleming The only thing that I can think of is to use the name of the containing directory. Does that seem like a reasonable convention? #2018-06-18 03:12:52 alexmiller We have talked about this a lot in the context of git deps. I don’t have time to write it all now will try to hit this tomorrow #2018-06-18 01:49:32 cfleming I could also allow a name to be provided in a :cursive/name my-module entry in the deps map, assuming that sort of thing is allowed. #2018-06-18 01:50:24 cfleming Oh, and one restriction is that modules have to have unique names. #2018-06-18 01:51:25 hagmonk I think the general sentiment has been to keep deps.edn uncluttered with anything not related to building a classpath or running the program. Whether unrelated keys are preserved seems to be by chance rather than design #2018-06-18 01:51:39 cfleming Ok #2018-06-18 01:52:03 hagmonk but you just need a name for the module, it doesn't need to be the same as, say, the official maven artifact name #2018-06-18 01:52:05 cfleming However, I doubt Cursive is the only system that would find it useful to assign a name to a project. #2018-06-18 01:52:30 cfleming True, but there’s literally nothing there to base it off right now. #2018-06-18 01:52:50 cfleming I mean, I could use the sha of the path to the project, but no-one is likely to be happy with that 🙂 #2018-06-18 01:53:30 hagmonk haha, what? who here doesn't love a full length SHA? :) #2018-06-18 01:54:27 cfleming Hey, it’s (almost) guaranteed unique! #2018-06-18 01:56:09 cfleming As I understand it, if there are two projects which both refer to a common library using a local ref, there’s no requirement even for the name they assign to it to be unique. #2018-06-18 01:56:54 cfleming i.e. project1: foo/bar {:local/root "../lib"} and project2: baz/bup {:local/root "../lib"} #2018-06-18 01:57:50 cfleming In fact, I have no idea where those names are used, if anywhere #2018-06-18 01:58:14 hagmonk I suppose the resolved path to the dep is what actually matters there, rather than the symbol. I'm not aware of any use for it either #2018-06-18 01:58:50 hagmonk I guess one could also look at any git metadata in the project folder - perhaps prefer that, then fall back to directory name #2018-06-18 01:59:17 cfleming I guess I could use the path from the main project root as the module name, but that will get weird if the local root is outside the main project. #2018-06-18 01:59:40 cfleming I don’t know if IntelliJ has restrictions on what modules can be called, actually. #2018-06-18 02:00:15 cfleming The problem with git metadata is that Cursive then needs git capabilities as well as maven capabilities #2018-06-18 02:00:37 hagmonk plus it won't always exist #2018-06-18 02:00:42 cfleming Right. #2018-06-18 02:01:26 hagmonk perhaps the basename of the resolved path of the dep is good enough for most cases? If there is a name clash it might be worth foisting that on the user anyhow #2018-06-18 02:01:55 cfleming That’s basically the name of the containing directory, right? #2018-06-18 02:02:12 hagmonk Right - or even "basename-1/basename" ... most of my projects would be sensibly named with that convention #2018-06-18 02:02:24 cfleming I guess I’m worried about cases like server/api/deps.edn and client/api/deps.edn #2018-06-18 02:02:32 cfleming Yeah #2018-06-18 02:02:58 cfleming I’ll see if I can find out whether there are restrictions on module names. #2018-06-18 02:03:20 cfleming I think I’ll just have to implement things until people stop complaining. #2018-06-18 02:04:09 hagmonk That is the zen of software development right there! #2018-06-18 02:04:30 cfleming The problem with foisting it on the user is that there’s no way to store a customisation for the name within the IntelliJ project - I have to return a project structure based on a list of deps.edn files, no more. #2018-06-18 02:04:52 cfleming Indeed - I think of it as a dynamic search algorithm. #2018-06-18 02:05:45 cfleming @alexmiller I’m interested if there’s an official position on keys like :cursive/name my-module in deps files. #2018-06-18 03:01:03 seancorfield @cfleming I don't know how "official" it is but whenever the subject has come up, Alex has always discouraged people adding "random" keys to the file... #2018-06-18 03:05:00 hagmonk Also anecdotal, but listening to Rich on the cognitect podcast (when will they fix the audio?) I heard him say how nice it was to look at a deps.edn file and see “only deps” in there … #2018-06-18 03:05:01 cfleming I guess I can always have a cursive.properties file alongside the deps.edn, but that seems way uglier to me. #2018-06-18 03:05:26 cfleming Well, except it’s not only deps. #2018-06-18 03:05:45 cfleming I mean, there are tasks with aliases etc etc as well. #2018-06-18 03:05:55 cfleming Anyway, I’ll wait and see what Alex says. #2018-06-18 03:07:32 hagmonk That was my thought too, I think it’s more accurate to say it contains “keys required to build a classpath and invoke the JVM with that classpath” #2018-06-18 03:11:38 seancorfield There's an argument that Leiningen's project.clj sets a precedent for allowing additional tool-specific keys... #2018-06-18 03:15:36 hagmonk I think tools.deps is a deliberate move away from that - where is the sweet spot though? Should the canonical name for the project be something in the deps file? It’s not really a tool-specific thing at all. Ironically, the deps file is a list of canonical names for other projects, or at least the names they go by in their repos #2018-06-18 03:21:03 seancorfield I would say that deps.edn goes a long way toward removing the importance of names -- you can give arbitrary names to local and Git deps. The names of Maven deps really only exist to match up with group/artifact names that are published. When you depend on, say, a Gist, or some code in a random directory, the name is unimportant and synthetic for the most part... #2018-06-18 03:22:58 cfleming Sadly, my users are not going to want shas as their module names. #2018-06-18 03:25:58 cfleming They’re fine in a gist link, but not for a descriptive name for someone who wants to get an overview of their project. #2018-06-18 03:33:01 hagmonk These are the kind of highfalutin customers @cfleming's software attracts :) #2018-06-18 03:36:41 cfleming They’re so fussy. #2018-06-18 03:39:38 hagmonk For some reason I'm reminded of Ghostbusters: "I've worked in the private sector … they expect results" #2018-06-18 03:41:53 cfleming It does sometimes help to imagine myself as Bill Murray when I have to do this stuff. #2018-06-19 04:26:15 cfleming Is there a list of likely locations I can try to look for the clojure command, so I can default it for the user if it exists? #2018-06-19 04:26:32 cfleming Currently my list only contains /usr/local/bin/clojure from OSX. #2018-06-19 04:27:53 cfleming Where is it on linux or WSL? #2018-06-19 04:31:50 cfleming @richiardiandrea Thanks. The problem is that when I execute from within IntelliJ, the user’s path is generally not set correctly. #2018-06-19 04:32:22 cfleming So the user will have to configure it, but I was hoping for a list of likely absolute paths I could check. #2018-06-19 04:33:32 richiardiandrea The one you mentioned is most likely one. I use $HOME/.local/bin/clojure #2018-06-19 04:33:46 richiardiandrea Some distro might install in /opt #2018-06-19 04:34:33 richiardiandrea Some directly in /usr/bin maybe? I guess these are the most common if I were to answer that #2018-06-19 04:42:04 cfleming How many installers for linux distros are out there at the moment? #2018-06-19 05:17:47 seancorfield @cfleming Why is the user's path not set for IntelliJ? (genuinely curious -- I'm not used to programs running without the standard user shell environment being available... I think?) #2018-06-19 06:43:40 dottedmag @seancorfield I have stumbled upon it on Mac: Dock and Finder are launched directly by user's launchd, which is launched directly by system launchd, so no shell is executed or profile file is read, so their environment is pretty bare. #2018-06-19 06:44:14 dottedmag @cfleming /opt/local/bin/clojure for MacPorts #2018-06-19 06:46:41 dottedmag @cfleming /usr/bin/clojure on Arch Linux (and I suppose in every other distro that bothered to package Clojure) #2018-06-19 06:48:33 dottedmag @cfleming /usr/local/bin/clojure for Linux installer from http://clojure.org #2018-06-19 07:45:13 cfleming @dottedmag Thanks! #2018-06-19 07:46:49 dominicm void places it in /usr/bin/clojure too #2018-06-19 09:27:04 cfleming @dominicm Is that on linux? #2018-06-19 09:39:17 dominicm @cfleming yes #2018-06-19 11:40:29 mfikes Given the above, it would ultimately end up having defaults like this on non-Windows platforms
["/usr/local/bin/clojure" "/usr/bin/clojure" "/opt/local/bin/clojure"]
#2018-06-19 12:12:44 rickmoynihan Yeah I’d much rather of not relied on the executable at all (and just used the library); but then you don’t know where the find the :install deps.edn file #2018-06-19 12:43:32 alexmiller I’m considering moving that into tools.deps #2018-06-19 16:43:04 rickmoynihan so it would be on the classpath? #2018-06-19 16:45:45 dominicm The risk with that I guess is that organisations cannot specify repos globally. Maybe the root directory should go into /etc/clojure.conf or something. That's what apache, nginx, etc. do #2018-06-19 17:02:16 alexmiller it would be on the classpath or perhaps it would be automatically included such that it isn’t even a distinct deps.edn. still tbd. #2018-06-19 17:03:08 alexmiller @dominicm I think that would be the right solution to that problem, if we wanted to do something about it (which I do not at this time) #2018-06-19 23:02:15 cfleming @rickmoynihan Yes, I would much prefer that too - then the user could just specify a tools.deps version. #2018-06-19 23:02:50 cfleming @alexmiller Moving the install file into tools.deps would be great. #2018-06-19 23:03:22 cfleming One possible source of confusion with that is that the versioning is different for tools.deps and the cli stuff. #2018-06-19 23:20:35 cfleming @mfikes Thanks - I looked through the lein-tools-deps issues when researching this but hadn’t found the code. #2018-06-20 16:09:04 gfredericks I just encountered my first instance of a clojure library only being available via github #2018-06-20 16:11:17 nwjsmith ? #2018-06-20 16:14:07 gfredericks i.e., it's not on clojars #2018-06-20 16:14:16 gfredericks you are expected to use it by specifying the github repo and a sha #2018-06-20 16:53:54 seancorfield The Cognitect Labs test-runner is like that -- but I assume you mean "aside from clj-specific tooling"? #2018-06-20 16:54:25 seancorfield (also my clj-new is not on Clojars but, again, clj-specific tooling) #2018-06-20 18:32:27 kenny Can I specify a username and password for a Maven repo? #2018-06-20 18:57:47 seancorfield @kenny In settings.xml I believe... #2018-06-20 18:58:08 kenny Not in a deps.edn? #2018-06-20 18:58:36 seancorfield You certainly don't want your Maven creds in your project's deps file under version control. #2018-06-20 18:58:49 kenny Via environment variables. #2018-06-20 18:59:10 seancorfield deps.edn is static EDN, no variables. #2018-06-20 18:59:49 kenny So there is no way to use private Maven repositories? Or does tools-deps read in settings.xml? #2018-06-20 19:01:36 seancorfield I haven't tried it myself (and haven't seen specific documentation about it). #2018-06-20 19:14:17 seancorfield Looks like it relies on Maven's standard settings.xml file according to the patch in that ticket. #2018-06-20 19:15:43 kenny Got it, thanks! #2018-06-20 19:54:13 alexmiller yes, that is all correct #2018-06-20 19:54:38 seancorfield (it was a TIL moment for me so thank you for asking @kenny!) #2018-06-20 19:54:59 alexmiller For Maven deps in authenticated repositories, existing Maven infrastructure is used to convey credentials via the ~/.m2/settings.xml:
<servers>
…
<server>
<id>my-auth-repo</id>
<username>zango</username>
<password>123</password>
</server>
…
</servers>
Then in your deps.edn include a repo with a name matching the <id>:
{:deps
{authenticated/dep {:mvn/version "1.2.3"}}
:mvn/repos
{"my-auth-repo" {:url ""}}}
#2018-06-20 20:28:25 seancorfield Thanks. I'd searched for settings.xml earlier and couldn't find anything in the guides or reference. #2018-06-20 20:58:30 alexmiller I forgot to add it #2018-06-21 00:44:57 kenny Thanks! #2018-06-20 19:55:54 hagmonk is it not possible to specify relative paths in :local/root? #2018-06-20 19:56:12 alexmiller it is possible #2018-06-20 19:56:33 hagmonk I'm trying to use lein-tools-deps to point to two different deps.edn, one of which points at the other via relative path #2018-06-20 19:56:48 hagmonk could be a lein-tools-deps bug … #2018-06-20 19:57:18 alexmiller there are some known issues with local to local and relative paths #2018-06-20 19:57:23 alexmiller particularly around :paths #2018-06-20 19:58:46 cfleming @alexmiller Just a gentle reminder about the module naming issue, you said you had had discussions about this in the context of git deps. #2018-06-20 20:01:16 alexmiller yeah, haven’t had time, and won’t today. but I haven’t completely forgotten #2018-06-20 20:01:26 cfleming Ok, thanks #2018-06-20 20:02:38 hagmonk changing :local/root to be an absolute path resolved things for lein-tools-deps. Hum. #2018-06-20 20:09:31 hagmonk Oh cute, I got it working #2018-06-20 20:10:09 hagmonk Solution is to not specify the deps.edn files directly in project.clj, just leave it as :project. Then, make an uber deps.edn that points at the deps.edn files. Things resolve as expected #2018-06-21 22:28:39 rickmoynihan Glad you got this working; if you think it’s a bug though please file an issue with lein-tools-deps
As Alex says, there’s some open bugs on tools.deps itself to do with paths, so could be that; but could also be due to us resolving paths relative to :project… would need to have a test case to recreate #2018-06-21 15:21:03 timgilbert Say, is there a tools.deps equivalent of lein deps, meaning "download everything you need to so I can cache it on my CI server"? #2018-06-21 15:26:27 bronsa clj -Sforce < /dev/null should do it #2018-06-21 15:29:48 sparkofreason I'm running most of my dev CLI stuff under WSL, but Cursive runs under Windows. So a tool like lein-tools-deps won't work correctly in the Cursive context. Anybody have any luck working around this? #2018-06-21 15:44:27 richiardiandrea Oh. I hoped that would work for a couple of colleagues of mine. I have exactly the same setup for macos users #2018-06-21 16:02:57 hagmonk I think lein-tools-deps works by invoking the clj command? If so you’d need to wait for Windows port. #2018-06-21 16:06:26 sparkofreason I can get clj to execute under WSL using a batch script, but not sure that will be sufficient. #2018-06-21 16:15:30 seancorfield @U06HHF230 would have more insight but I would have expected lein-tools-deps to work on Windows if you explicitly specify which deps.edn files you need to use in project.clj so it didn't have to rely on tools.deps.alpha's ability to shell out to the clojure command? #2018-06-21 16:43:01 richiardiandrea I think lein-tools-deps calls clojure under the hood, but I saw that you can configure the location of it as well. #2018-06-21 17:13:46 seancorfield Well, if it asks tools.deps.alpha for the (clojure-env), the shell out to clojure happens there (inside tools.deps.alpha itself). When I switch development machines (fairly soon, I suspect), I'll also be dealing with this since I use Atom/ProtoREPL on the host O/S. Right now, I'm on a Mac so the host for Atom is the same as the host for lein/`boot`/`clj` but I've already run into this on (Windows) laptop and my workaround was to start a REPL under WSL and then connect to it from Atom -- but there are still gotchas like file paths differing between Windows and WSL: when you "load file", ProtoREPL assumes that the REPL can be asked to load the file using the editor host path which fails unless you've been very careful setting up symlinks in WSL etc. #2018-06-21 22:45:44 rickmoynihan Hi, sorry I don’t use Windows or have access to a machine I can test this setup on. Am I right that the problem is that you’re effectively running clojure in a (pseudo) linux environment, but Cursive in Windows; so the paths are resolving incorrectly?
@U04V70XH6 is right that we call clojure-env and rely on tools.deps shelling out to clojure to get the env for us. Unfortunately it’s a slightly tricky problem just now and the current design is a bit brittle.
I’d be open to disabling the call to clojure-env; though if we do that you’d also need to set the paths to your :install deps.edn and user one by hand.
If this issue is effecting you though, please open an issue where we can discuss more. #2018-06-22 00:53:18 cfleming I was actually planning to ask about this. It is possible to access WSL from Windows? If I try to open /usr/local/bin/clojure or whatever from Windows, does that work or not? #2018-06-22 00:54:57 cfleming My currently plan for Cursive is to bypass the CLI stuff altogether, and just use the library. There are some outstanding questions about how to do that, such as how to get the system deps.edn - for an initial beta I’m going to bundle it with Cursive, but I’m hoping that it will eventually be available in the lib - Alex mentioned he was thinking about that. #2018-06-22 00:55:24 cfleming Assuming I just use the lib, then it should all work on Windows. #2018-06-22 00:56:47 cfleming Another complication is that the user will have to choose the version of t.d.a to use, and those versions don’t correspond to CLI versions, so there’s some potential for confusion. #2018-06-22 00:57:45 cfleming And I guess if people are using the CLI in WSL and Cursive can’t access that, then that will also be confusing since you’ll end up with two parallel systems. #2018-06-22 01:01:39 cfleming The red letters are:
> DO NOT, under ANY circumstances, create and/or modify Linux files using Windows apps, tools, scripts, consoles, etc.
And some following large black ones:
> Creating/changing Linux files from Windows will likely result in data corruption and/or damage your Linux environment requiring you to uninstall & reinstall your distro! #2018-06-22 01:02:05 cfleming So, I read that to mean that using the CLI from WSL will be totally inaccessible to Cursive anyway. #2018-06-22 01:02:23 cfleming @U066LQXPZ ^^ #2018-06-22 01:10:28 sparkofreason I think that guidance is somewhat out of date, and the interop between WSL and Windows file systems is much improved. I actually keep all the clojure project stuff on the Windows side anyway, but and do CLI stuff from WSL. #2018-06-22 01:13:59 cfleming Right, but I think that WSL is aware of Windows but not the other way around. #2018-06-22 01:14:15 cfleming Although I haven’t really followed it recently. #2018-06-22 01:14:17 sparkofreason It is possible to execute clojure under WSL from Windows, bash -c "clojure". I haven't been able to make that work with lein-tools-deps yet, was planning on making a fork and investigating locally. #2018-06-22 01:15:06 cfleming I’d be selfishly interested to know if you could execute it via ProcessBuilder from a Java app running on Windows. #2018-06-22 01:16:15 cfleming If that is possible, then it should be possible to access it from Cursive. I’m still probably going to go lib-only unless there’s a really good reason not to because it’s much simpler, but it would be interesting to know. #2018-06-22 01:18:58 cfleming @U066LQXPZ BTW that guidance is still in the WSL FAQ: https://docs.microsoft.com/en-us/windows/wsl/faq
> One of the main limitations of using WSL is that changing Linux files with a Windows app or tool is not allowed. #2018-06-22 01:22:15 cfleming Comment here: https://superuser.com/a/1111092
> Every file that resides on a Volfs folder (like /home) has Extended Attributes that store the Linux permissions of that file. If you edit that file on a windows editor then those attributes are lost and file will disappear from bash. #2018-06-22 01:31:47 cfleming @U066LQXPZ If you’re running your dev in WSL, doesn’t that mean that your source files are also under WSL? How does IntelliJ access them? Or do you treat WSL as a server you deploy to? #2018-06-22 01:41:01 sparkofreason All of the files are in the windows FS. The tooling (apart from the IDE) executes under WSL, and can work with the windows FS no problem. #2018-06-22 01:41:42 cfleming Oh right, of course. So your Maven local repo etc is all under Windows? That makes sense. #2018-06-22 01:42:50 cfleming So I guess the only issue would be accessing the installed deps.edn files (system and user). Do you use the CLJ_CONFIG env var to keep the user one under Windows too? #2018-06-22 01:51:04 sparkofreason No, that is on the todo list. #2018-06-22 01:52:00 sparkofreason It's how I have both lein and boot set up. #2018-06-22 02:19:51 sparkofreason @U0567Q30W (clojure.java.shell/sh "clojure.bat" "-Sdescribe") works. I assume ProcessBuilder would as well. #2018-06-22 02:20:18 cfleming @U066LQXPZ Ok, interesting, thanks. #2018-06-22 02:21:52 cfleming I guess the hard thing is that the paths that -Sdescribe will return are all in linux-land. So even if you configure CLJ_CONFIG to /mnt/c/Users/dave/.clojure or something, that path is still invalid in Windows. #2018-06-22 02:25:37 sparkofreason Right, you'd definitely want to move it to be under windows. You can configure WSL to behave as though Windows drives are mounted under root, so you'd get /c/Users/dave/.clojure, but obviously that still can't be used as-is. #2018-06-22 02:33:16 cfleming I think it would be better to leave them under /mnt, since Cursive has to be aware it’s talking to WSL anyway. #2018-06-22 02:33:33 cfleming It would make it easier to detect that it’s getting a Windows path back from Linux #2018-06-22 03:17:59 sparkofreason Yeah, but other things like docker benefit from this (so they tell me, I haven't run into this myself). Maybe some sort of config option to indicate WSL? #2018-06-22 03:37:20 seancorfield Microsoft Windows [Version 10.0.17692.1000]
(c) 2018 Microsoft Corporation. All rights reserved.
C:\Users\seanc>wsl.exe clojure
Clojure 1.9.0
user=>
C:\Users\seanc>wsl.exe clojure -Sdescribe
{:version "1.9.0.381"
:config-files ["/usr/local/lib/clojure/deps.edn" "/home/seanc/.clojure/deps.edn" ]
:install-dir "/usr/local/lib/clojure"
:config-dir "/home/seanc/.clojure"
:cache-dir "/home/seanc/.clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
C:\Users\seanc> #2018-06-22 03:37:39 seancorfield Use wsl.exe to invoke Linux commands 🙂 #2018-06-22 03:39:27 seancorfield And just to be clear, when you do that, the Linux command executes in the context of Linux, e.g., user.home: C:\Users\seanc>wsl.exe clojure
Clojure 1.9.0
user=> (System/getProperty "user.home")
"/home/seanc"
user=>
#2018-06-22 04:01:49 seancorfield (of course this totally doesn't help in terms of actually accessing those files from Windows, nor in starting/running a REPL that can accept "load file" commands from Windows) #2018-06-22 04:02:09 cfleming @U04V70XH6 Thanks. So I’d need to make Cursive WSL-aware. #2018-06-22 04:02:31 cfleming The problem is also still accessing the deps files from Windows. #2018-06-22 04:02:42 cfleming (or, a problem, rather) #2018-06-22 04:03:17 seancorfield Yeah, it's a tough nut to crack. I've dinked around a bit with Windows-Atom/ProtoREPL and WSL-REPL but not really gotten anywhere. #2018-06-22 04:03:53 cfleming I think just using the lib and keeping everything in Windows-land looks like the cleanest solution. #2018-06-22 04:04:45 seancorfield Well, once the lib knows how to -Sdescribe the environment... #2018-06-22 04:05:42 cfleming Right, I’d have to hack that for the moment, probably bundle the system deps.edn in Cursive, and perhaps allow the user to edit it somehow. #2018-06-22 04:06:23 cfleming And allow them to configure where their user one is, defaulting as required. #2018-06-22 04:06:42 seancorfield My fallback with Boot has been to have Boot installed on both Windows and WSL and let Atom start a REPL via Boot and keep all the files in a project structure under the C: drive that I can easily map to Linux (symlink to make the same file paths without /mnt/c). #2018-06-22 04:09:47 seancorfield When the PowerShell port of clojure hits, we'll all be a lot better off -- but we'll still have the weird "duplication" of the "home" deps.edn because that will be in a different place on Windows and WSL. I can't see a workaround for that since the user home folder has to be different. #2018-06-22 04:16:52 cfleming You could use CLJ_CONFIG in WSL? #2018-06-22 04:17:09 cfleming And point the linux one to where it is on Windows? #2018-06-22 04:23:50 seancorfield I haven't tried CLJ_CONFIG... #2018-06-22 04:26:01 seancorfield Oh right, so on WSL, CLJ_CONFIG could point to /mnt/c/Users/<whatever>/... yeah, sounds like a good approach once we have PowerShell clojure. #2018-06-22 04:38:51 cfleming Right. #2018-06-22 04:39:57 cfleming Although generally I think the best option for clients like Cursive and lein-tools-deps would be bundling the system deps.edn in the lib, and perhaps providing a method to locate the user deps.edn. #2018-06-22 04:54:13 seancorfield When I first built boot-tools-deps I baked in the deps.edn from the brew-install GitHub repo so I didn't need clojure installed on the system. Alex regularly told me that was a bad idea 🙂 So I changed it to call (clojure-env) once t.d.a had that API. And now we're talking about the system deps.edn being baked into t.d.a anyway! :rolling_on_the_floor_laughing: #2018-06-22 10:53:29 rickmoynihan Interesting discussion - I think I’d like to remove the call to clojure-env from lein-tools-deps and follow the bundled system approach; and just let people override the :system location if they need to. #2018-06-22 19:52:46 cfleming @U06HHF230 Yes, I agree - this would make Cursive much easier too #2018-06-22 19:53:03 cfleming The only issue is the need to specify the version to use. #2018-06-21 15:33:37 alexmiller @timgilbert additionally, you could set :mvn/local-repo to a fresh directory, then capture that directory to know what you need. #2018-06-21 17:12:10 timgilbert Am I crazy, or is tools.deps super slow at downloading versus lein deps? I'm seeing my download step go from about 40 secs to about 4 minutes on the same hardware #2018-06-21 17:43:26 alexmiller I haven’t heard that before now #2018-06-21 17:44:36 alexmiller Are you comparing with cold m2 repo in both cases? #2018-06-21 17:47:30 timgilbert Yeah, they are both running from nothing on circleci servers. I do wonder if it might have to do with invalid artifacts in my tree, combined with a private S3 repo. #2018-06-21 17:48:31 timgilbert I will do some poking around and file a ticket if I can come up with some numbers and a reproducible case #2018-06-21 17:50:25 alexmiller Yeah, would be curious #2018-06-21 17:50:59 alexmiller Since most people have stuff in cache, most people wouldn’t notice #2018-06-21 17:52:42 timgilbert Yeah. Does tools.deps cache misses? Somehow or another I've wound up with com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml in my deps tree, which doesn't exist, and it seems to be causing some of the latency along with several other similar artifacts #2018-06-21 17:53:50 timgilbert (Obviously, these are things I am going to exclude from the tree once I've tracked them down) #2018-06-21 18:09:32 alexmiller tools.deps uses the same Maven apis that lein (via pomegranate) does to download metadata and deps #2018-06-21 18:11:02 alexmiller so I would expect it to generally do pretty similar things. there are some differences, notably that tools.deps uses its own transitive dep walk and can potentially choose different versions and may walk versions that lein deps doesn’t. #2018-06-21 22:48:41 rickmoynihan I’ve been meaning to ask are the differences in resolution documented anywhere, or formally known? Asking because it’s relevant to leiningen projects moving deps into deps.edn and lein-tools-deps. #2018-06-22 00:54:51 alexmiller Well it’s known to me but I haven’t gotten around to writing it up yet #2018-06-22 00:55:18 alexmiller It’s somewhat similar to the new golang stuff #2018-06-21 18:11:33 alexmiller tools.deps itself doesn’t explicitly do any caching of metadata or libs at all - Maven handles all of that #2018-06-21 18:13:11 alexmiller several possible things to investigate there. if you file a ticket with a deps.edn to repro, I would eventually take a look at it #2018-06-21 18:13:54 timgilbert Ok, cool. I'll see what I can do to come up with a reproducible case. Thanks! #2018-06-21 21:45:40 kenny If both :sha and :tag are provided for a Git dep but :tag points to a different :sha, which SHA is used? #2018-06-21 21:56:46 seancorfield @kenny I thought only :sha was used for dependencies and the :tag was there for use with the option that adds :sha based on :tag...? #2018-06-21 21:56:58 seancorfield (it's easy enough to test the behavior!) #2018-06-21 21:58:17 kenny Also makes it sound like :sha is always required. #2018-06-21 21:59:09 kenny Yes. Will test it out later today. Definitely could use some clarification in the docs though. #2018-06-21 22:04:34 seancorfield That reads that :sha is used for dependencies and :tag is purely descriptive -- although there's an option to update deps.edn adding/replacing SHAs with the values that correspond to tags: clj -Sresolve-tags #2018-06-21 22:06:04 seancorfield My interpretation of that is that if you have :sha and :tag and the latter doesn't match the former, it would not affect resolution. And if you ran clj -Sresolve-tags it would update the :sha to match what the :tag pointed at. @alexmiller can you confirm or deny? #2018-06-21 22:06:50 alexmiller Correct #2018-06-21 22:07:02 kenny That interpretation makes sense to me. I'd still suggest adding that information into the docs 🙂 #2018-06-21 22:11:50 alexmiller Still considering whether to remove :tag and -Sresolve-tags entirely :) #2018-06-21 22:19:27 kenny The reason I was thinking about using :tag is to convey breakage. With only :sha, determining breakage is a lot harder. #2018-06-22 01:11:08 alexmiller Can you explain? #2018-06-22 01:11:25 alexmiller If you mean like api breakage, just don’t do that :) #2018-06-22 01:12:03 alexmiller We have sha logic comparison in there - it can tell which one is a parent of the other #2018-06-22 01:12:58 alexmiller If more than one available in the tree, it will pick the descendant #2018-06-22 15:30:30 val_waeselynck It seems to me it's not possible to have a spaces in a :main-opts entry (e.g :main-opts ["--space-separated-arg" "Joe Jack Williams Averell"]), is this is known issue? #2018-06-22 15:32:32 alexmiller Yes, use commas for now #2018-06-22 15:33:04 alexmiller Or in a case like this, not going to work for now #2018-06-22 15:38:01 val_waeselynck @alexmiller thanks, currently my workaround is simply to pass the args form the command line #2018-06-24 04:40:17 hagmonk I've updated https://github.com/hagmonk/find-deps, it now allows you to save matches even if the deps.edn file doesn't exist (handy for new projects). It also has a new -F:cli output format that prints the deps on a single line appropriate for clojure -Sdeps usage #2018-06-24 08:58:20 pesterhazy @hagmonk just tried it, it doesn't seem to work for me. After following the instructions, clj -A:find-deps -h gives me the normal "deps" help screen #2018-06-24 09:00:12 pesterhazy is the sha in the readme up to date? #2018-06-24 13:03:29 alexmiller Do you have the latest clj? That -h thing was a bug I fixed a while back #2018-06-24 14:29:23 pesterhazy @alexmiller I have version = 1.9.0.381 from homebrew #2018-06-24 14:31:46 alexmiller That should be latest #2018-06-24 14:33:23 alexmiller Maybe you’d see that if it’s missing the find-deps alias #2018-06-25 17:44:41 hagmonk I've pushed a new find-deps with @pesterhazy's PR merged and a fix for no longer returning SNAPSHOT versions from clojars (hard coded for the time being) #2018-06-25 17:48:19 pesterhazy @hagmonk just tried it, works really well #2018-06-25 17:48:37 pesterhazy I think hardcoding the hide-SNAPSHOT feature is a much better default #2018-06-25 17:49:41 hagmonk Yes definitely, I think the bug should not crop up in maven searches, I need to spend time finding an example there to validate that #2018-06-25 17:49:58 hagmonk I believe maven doesn't offer snapshot results by default #2018-06-25 17:53:22 pesterhazy good job on the library btw! #2018-06-25 17:59:39 hagmonk Thanks! Let me know if any of the search results don't look good :) #2018-06-25 18:54:06 dominicm @hagmonk one piece of feedback is that it will modify the deps.edn formatting. #2018-06-25 18:56:56 hagmonk @dominicm that's worth an issue on github … appreciate any pointers you might have for how to avoid that #2018-06-25 18:57:25 dominicm Pack avoids it (my library). Happy to try share a library for solving this problem. I'm secretly hopeful it can go into tools deps itself. #2018-06-25 19:27:33 hagmonk Thanks @dominicm, I'll have a look after work #2018-06-25 19:28:20 hagmonk I'm still enjoying how much time this is saving me, occasionally I'm surprised it works as well as it does 😂 #2018-06-25 19:28:33 hagmonk clj -A:find-deps "clojure cache"
{:deps {org.clojure/core.cache {:mvn/version "0.7.1"}}}
#2018-06-25 19:31:35 pesterhazy one possibility would be to run the result through zprint, a great library for re-formatting clojure code or edn #2018-06-25 19:32:41 pesterhazy zprint is an under-appreciated gem #2018-06-25 19:55:35 dominicm @pesterhazy the problem is that everybody does formatting a little differently. I took the approach of trying to match existing indentation at most levels. #2018-06-25 20:12:01 mkvlr it would be so great if we could agree on a canonical way to format clojure code #2018-06-25 20:13:47 mkvlr having worked with elm-format and exfmt I found it so nice to just not have to worry about this at all anymore #2018-06-25 20:14:20 pesterhazy zprint gets you 90% there #2018-06-25 20:16:18 dominicm Zprint puts my defn all on one line, which feels wrong #2018-06-25 20:23:36 pesterhazy They are too short :) #2018-06-25 20:24:42 dominicm Do you happen to know of a way to change that behaviour? 80 chars isn't a goal, so much as a limit. #2018-06-25 20:27:01 pesterhazy Not off the top of my head #2018-06-25 20:29:04 pesterhazy But if you get consistency, convenience and the end of style wars in return, wouldn’t oneliners be a small price to pay? #2018-06-25 20:30:24 dominicm I want consistency, very much. But I first have to convince important people that they will like the formatting, and that it's worth pioneering. #2018-06-26 07:50:06 kommen it says it is meant for vectors, but might work for lists as well #2018-06-26 07:51:19 dominicm I don't necessarily want to have it retain where I place newlines. I just find it weird to see:
(defn foo [args] body)
over
(defn foo
[args]
body)
#2018-06-26 07:59:03 kommen ah, I see. haven’t tried to configure that, but pretty sure it is possible #2018-06-26 08:04:09 dominicm It would be nice to have a style which is "do what most libraries do for formatting" #2018-06-26 08:07:37 kommen I think the :community style is meant for that. kkinnear has been open to adapt that style to fit best practices, especially if those things would be documented in the clojure community style guide #2018-06-26 08:50:50 pesterhazy NOT respecting newlines is one of the best things about elm-format #2018-06-26 08:51:15 pesterhazy you can totally write your code in a single line and elm-format will expand it out #2018-06-26 08:51:23 pesterhazy it helped me a lot when I was learning the language #2018-06-26 08:55:19 dominicm I particularly don't want my tool to look at the input beyond raw clojure forms tbh. I'd even be happy if it made decisions about #:foo {:blah 10} into {:foo/blah 10} (but that's because I don't like the syntax) #2018-06-25 20:14:46 pesterhazy Agree about elm-format, it’s a pleasure to use #2018-06-25 20:14:47 mkvlr we’re using zprint but there’s so many knobs. I think an autoformatter should ideally have 0 options. (or maybe one for the line width) #2018-06-25 20:15:06 mkvlr sorry for going a bit offtopic here… #2018-06-25 21:57:24 lwhorton i’m not sure if this is the right place to ask… but does the :git/url coordinate require a deps.edn file at that coordinate? #2018-06-25 21:57:55 lwhorton i.e. it doesn’t try to read a project.clj or boot file… it only plays nicely with another deps.edn (or maybe a pom) file? #2018-06-25 21:58:46 ghadi That's right @lwhorton #2018-06-25 21:58:59 ghadi It does read deps.edn and pom.xml transitively #2018-06-25 22:08:31 alexmiller correct (for now) #2018-06-25 22:08:56 alexmiller it’s also tolerant of finding none of those for the case of a dep with no transitive deps #2018-06-25 22:56:05 cfleming > I think an autoformatter should ideally have 0 options. (or maybe one for the line width)
Preach it! If I ever design a language, it will have a gofmt-style formatter from day 1. #2018-06-25 22:57:08 cfleming Indentation turns out to be one of those things that everyone thinks they care about, until they can’t control it, at which point it turns out they don’t care much as long as the results are reasonable. #2018-06-25 22:59:13 alexmiller +1 #2018-06-26 02:12:50 hagmonk Would love for us to swallow that frog and just settle on one format to rule them all. I'd gladly give up my personal formatting preferences in exchange for not having to care about this ever again :) #2018-06-26 06:44:58 pesterhazy If @bozhidar and @cfleming band together and settle on a zformat formatting style, that’s a significant portion of the market #2018-06-26 06:45:47 pesterhazy Make it opt-in and apply on save #2018-06-26 06:51:08 orestis We’re considering introducing Clojure to an existing team – not having to argue about formatting would be huge — zprint looks fantastic, with native-image support so could even be run on a github pre-commit hook. #2018-06-26 06:54:14 hagmonk I think people quite quickly get the whole paredit/parinfer concept, where the editor prevents you from creating an improperly delimited program … it doesn't seem a fair stretch to also prevent you from ever creating an improperly indented program #2018-07-02 08:46:31 rickmoynihan I’m definitely all for structured editing; a classic example of a really good one was Diva Scheme for PLT Scheme (now Racket):
https://www.youtube.com/watch?v=GnQV4je9wTQ
You need a mode to disable them temporarily though e.g. when resolving a merge. #2018-06-26 06:56:50 hagmonk I'm trying out parinfer and am irritated that I seem able to hit "tab" and move blocks of code around to create valid but aesthetically awful code (probably some config thing I borked). I'd love for it to be impossible to write untidy code, like making it physically impossible to create: #2018-06-26 06:57:42 hagmonk dangling paren:
(comment
(some-stuff "test")
)
or have two newlines between def:
(def this "thing")
(def other "thing")
… etc #2018-06-26 07:00:04 dominicm Parinfer + zprint could be quite great for beginners... #2018-06-26 07:16:20 orestis OTOH, I tried Parinfer via Nightcode — I knew how to write Clojure code in spacemacs via Cider, which doesn’t try to do too much — but parinfer messed me up quite a lot — it tried to be helpful but got in my way. #2018-06-26 07:35:01 dominicm have you tried it since smart mode was added? #2018-06-26 09:16:45 orestis I need to find how to integrate with spacemacs and evil-mode, but will try again… #2018-06-26 07:17:20 orestis It helps a beginner to be able to balance forms manually, esp. if you have a linter that will point out possible problems, and a formatter that would take stylistic decisions away from you. Parinfer might be useful once you make relatively few mistakes when coding Clojure. #2018-06-26 07:24:19 hagmonk Parinfer is a little more loose, I agree. I think we could go a step further and have editors that totally precluded interim mistakes at all. Like: trying to add a third argument to a 2-arity function. Why do we let you type that in? Why do we let you type in something that is highly likely knowable at the reader level to be invalid, such as transposing the arguments to map – i.e. putting something that doesn't support IFn as the first argument? #2018-06-26 07:26:26 hagmonk I realize the dynamic nature makes some things hard to guarantee, but even though I'd say I'm a journeyman Clojure programmer, I would still appreciate the help. It's like type checking but while you type, not when you compile. #2018-06-26 07:27:09 orestis The UX for that is extremely hard to get right though — when I type a character, I expect it to show up in my buffer. If the editor puts a squiggly line later, then I know something’s wrong and I can learn something from it. #2018-06-26 07:31:52 hagmonk It need not be "can't type the key" levels of strictness, more like how we flag unbalanced parens today … "this is for sure going to barf". Ideally unbalanced parens become physically impossible, while messing up arguments graduates to being detected in the editor buffer rather than runtime or at compilation #2018-06-26 07:35:31 hagmonk I think we also have opportunity (I realize this is drifting off tools.deps but ya'll are an erudite bunch) for hinting, for instance, when I type (let [[foo bar] some-thing] …) I occasionally pass in something non-seqable as some-thing and invariably it takes me a few minutes to reverse engineer the cryptic error message and wth I did to cause it. Just flagging in the editor "reminder: some-thing now has to be seqable!" could be quite interesting … #2018-06-26 07:42:21 cfleming Plus some static analysis #2018-06-26 07:42:36 cfleming This should probably get moved over to #editors though. #2018-06-26 07:45:48 hagmonk Yes, I recall this whizzing past at some point. I'll take another look #2018-06-26 10:45:08 mgrbyte having trouble figuring out what the correct coordinate would be to use a local jar (datomic-pro), anyone doing this? #2018-06-26 10:47:04 dominicm @mgrbyte :local/root pointed at the jar #2018-06-26 10:53:35 mgrbyte that's what I've tried. doesn't work for me. I have:
:deps {
com.datomic/datomic.api {:local/root "/home/matt/datomic-pro/datomic-pro-0.9.5697/datomic-pro-0.9.5697.jar"}
}
#2018-06-26 10:54:27 dominicm @mgrbyte I may have crossed wires, just to check, can you try :deps/root instead? #2018-06-26 10:54:59 mgrbyte It's definitely :local/root in the map in the docs #2018-06-26 10:55:09 dominicm ah, you're having an issue with that working 😄 #2018-06-26 10:55:31 dominicm @mgrbyte I'm not sure I understand, what is the problem? 🙂 #2018-06-26 10:55:42 mgrbyte clj works, but requiring the dep doesn't #2018-06-26 10:55:55 mgrbyte clj
Clojure 1.9.0
user=> (require '[datomic.api])
ClassNotFoundException org.fressian.handlers.IWriteHandlerLookup java.net.URLClassLoader.findClass (URLClassLoader.java:381)
user=>
#2018-06-26 10:56:27 mgrbyte I'm using the very same jar in a lein project, which works fine. #2018-06-26 10:57:29 mgrbyte installed locally via bin/maven-install in the datomic-pro distro (same path as the jar in the deps I listed above) #2018-06-26 10:58:42 mgrbyte nm, think I see what's going on here #2018-06-26 10:59:23 mgrbyte datomic pro has a pom.xml with a bunch of deps in it. can't just use that jar, need all the deps in "./lib" from the distro too. #2018-06-26 12:11:59 alexmiller The latest tools.deps should actually read the pom inside the jar and load those for you #2018-06-26 12:12:24 alexmiller Make sure you are on latest clj tool #2018-06-26 12:54:08 mgrbyte thanks, will try that now #2018-06-26 12:54:54 mgrbyte Is there any reason to prefer to doing that than using bin/maven-install and using the mvn/version w/datomic-pro? #2018-06-26 12:55:53 alexmiller you can do either. you can also use the clj support to load the pro jar from your authenticated s3 datomic repo #2018-06-26 12:57:44 mgrbyte ty #2018-06-26 11:05:47 dominicm I thought it might be that. #2018-06-26 11:06:01 dominicm You should be able to set :local/root to the dir with the pom.xml in, but not certain. #2018-06-26 11:45:05 mgrbyte nearly, fails with some dep. Not a problem tho, can just use :mvn/version of the local install #2018-06-26 11:54:06 hkupty Quick question.. does tools.deps expose a function to add to classpath? I couldn't find anything from the documents but I would also like to avoid adding pomegranate or lucidity.package.. #2018-06-26 11:56:20 borkdude @ingvij I think it’s called add-lib #2018-06-26 12:01:01 hkupty Not there as of latest published version.. #2018-06-26 12:01:01 alexmiller Still experimental #2018-06-26 12:01:39 hkupty I see.. So that can change until I actually can rely upon.. #2018-06-26 12:01:51 hkupty I'll wait then.. not in a hurry for that though #2018-06-26 19:11:19 hagmonk @alexmiller I was going to start thinking about add-lib in the context of find-deps. I could have users do this: (-> "clojure cache" find-deps.core/deps clojure.tools.deps.alpha.repl/add-lib) #2018-06-26 19:12:03 hagmonk or I could provide a fn inside find-deps.core that invokes add-lib for you. right now I'd have to depend on your add-lib branch, so I was hesitant to bake that in quite yet #2018-06-26 19:12:14 alexmiller yeah, I wouldn’t do that yet #2018-06-26 19:14:42 hagmonk cool, I might limit integration to some notes in the readme for the time being. I can't wait until we have a workflow where I can seamlessly add deps without restarting anything :) #2018-06-26 20:41:37 cfleming @hagmonk Have you ever tried alembic? #2018-06-26 20:56:00 hagmonk @cfleming I have not ... Googling ... #2018-06-26 20:57:02 hagmonk fascinating, this would have been nice to have :) #2018-06-26 21:42:24 alexmiller note that the tools.deps add-lib is a little different than the other existing things out there #2018-06-26 21:43:06 alexmiller it’s not just blindly adding a lib (and/or it’s transitive deps), it’s actually detecting which things you don’t already have, resolving lib versions, and updating an internal map of loaded libs and versions #2018-06-26 21:44:34 hagmonk and obviously it can work on more than just jars #2018-06-26 21:45:32 alexmiller that too, all the standard tools.deps differences #2018-06-26 22:09:55 cfleming Alembic also does the detection of existing things you have installed, although it doesn’t use a map, it uses the list of things in your existing classloader. #2018-06-26 22:48:23 alexmiller The tools.deps impl falls back to that #2018-06-26 22:31:41 hagmonk I have some crazy ideas in this space - I feel like we can go a lot further than just adding the libs. For find-deps I want to analyze clj code on github to build "recommended deps" based on the one you're importing. For instance, reagent and re-frame often go together. Intelligently figuring out exclusions could be another one. #2018-06-26 22:46:18 sparkofreason How would I do a local build and deploy of tools.deps.alpha? #2018-06-26 22:46:42 alexmiller mvn clean install #2018-06-26 22:47:46 alexmiller That will use the current snapshot version in the pom #2018-06-26 23:26:10 sparkofreason Debugging some issues using tools.deps.alpha on Windows, and pretty much a noob with aether. Why would clojure.tools.deps.alpha.util.maven/make-system return nil? #2018-06-26 23:26:45 alexmiller Dunno, not sure why windows would be special #2018-06-26 23:27:13 alexmiller I’ve got ports of clj using tdeps on Windows that work #2018-06-26 23:29:38 sparkofreason Hmmm, that's what I would have guessed. It's being called by lein-tools-deps, so maybe the problem originates farther up the call stack. #2018-06-26 23:41:14 seancorfield @dave.dixon Perhaps because lein-tools-deps calls (clojure-env) in t.d.a. and that shells out to call clojure to get the list of deps.edn files to consume -- and I would expect that command to fail on Windows (until we have a Windows port of the clj/`clojure` shell scripts). #2018-06-26 23:41:58 seancorfield (so maybe it doesn't have the environment that make-system requires) #2018-06-26 23:42:46 sparkofreason @seancorfield That I have working, albeit locally hacked. I'll double-check that the right info is getting scraped. #2018-06-26 23:42:56 seancorfield Ah, OK. Cool. #2018-06-26 23:43:15 seancorfield I nearly said "...unless you have a mocked up version of clojure on Windows!" 🙂 #2018-06-27 01:13:08 sparkofreason Okay, wacky problem 1 solved, somehow I managed to have two versions of lein installed. #2018-06-27 16:22:36 sparkofreason Thanks to everyone's help, I was able to get a working setup using lein-tools-deps and Cursive on Windows, with clj installed on WSL. It's pretty hacky, but I think will tide me over until the tooling solves this problem in the right way. See fork on lein-tools-deps at https://github.com/ComputeSoftware/lein-tools-deps. Use at your own risk 😋 #2018-06-28 10:04:32 darnok I'd like to use :local/root to tie up my dependent projects in the monolith repo, but I have an issue with one of the dependencies. It has one Java source file. Can you recommend, how to handle this? #2018-06-28 11:33:15 alexmiller Well one solution is to move it into its own project and make an artifact #2018-06-28 11:34:52 darnok ok #2018-06-28 11:37:42 darnok I thought that it would need to be resolved like this. Thanks 🙂. #2018-06-28 14:00:47 dottedmag #2018-06-28 14:01:02 dottedmag @alexmiller ^ is this an intentional behaviour? #2018-06-28 14:20:44 alexmiller there’s a ticket, send me a patch :) #2018-06-28 14:21:09 alexmiller I just haven’t gotten to it yet #2018-06-28 14:42:05 pesterhazy yeah, the "problem" I reported a few days back was because the alias wasn't properly configured; a simple name check would caught this #2018-06-28 14:49:53 alexmiller happy to fix #2018-06-28 14:50:51 dottedmag @bhauman ^ #2018-06-28 14:53:50 bhauman I think my use of the word extremely was extremely extreme #2018-06-28 15:41:46 seancorfield It's bitten me a couple of times so I'll vote that ticket 🙂 but it hasn't bitten me hard/often enough for me to summon up the energy to create such a ticket... #2018-06-28 16:20:44 hagmonk today in #tools-deps: extremely extreme bugs! do they bite hard and often enough for a ticket? do they come from the Australian mainland? only jira can tell! #2018-06-28 16:20:55 hagmonk (clearly this channel is where all the action is at) #2018-06-28 16:36:31 alexmiller vote away #2018-06-28 16:36:40 alexmiller if no else patches I’ll get to it when I next work on TDEPS #2018-06-28 16:40:07 bhauman Thanks Alex! #2018-06-28 16:46:38 seancorfield Yes, thank you, Alex! #2018-06-28 21:07:04 kenny Why is the clojure CLI written in bash instead of Python or Node? #2018-06-28 21:07:27 dominicm @kenny because shell is universally available and python/node isn't. #2018-06-28 21:08:51 dominicm Hopefully it can be ported to POSIX shell without any bash, that would be a nice improvement 🙂 #2018-06-28 21:09:51 kenny Why do you say Python and Node aren't universally available? #2018-06-28 21:11:27 dottedmag {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "7f0d10100b3f1e1d1a13"}, :content ("[email protected]")} #2018-06-28 21:11:29 dominicm @kenny because they're less likely to be installed than bash. #2018-06-28 21:12:46 dottedmag Even if they were, what'd be the point? clojure(1) is trivial. #2018-06-28 21:15:07 kenny Installing Python/Node is trivial. Point is you get a script that is portable to any OS. #2018-06-28 21:17:13 dottedmag @dominicm clojure uses arrays from Bash. Not trivially convertable. #2018-06-28 21:18:30 dottedmag Also that means testing the script under several implementations (busybox, dash and bash at least). Not worth it. #2018-06-28 21:22:34 dominicm I personally am ready to remove bash for all but clojure on my system. I use dash & fish primarily. #2018-06-28 21:22:37 alexmiller because I wrote it and I don’t like Python or JS :) #2018-06-28 21:22:51 dominicm @alexmiller So 💎 ruby? #2018-06-28 21:22:57 alexmiller geez man #2018-06-28 21:23:37 alexmiller for whatever reason, I just don’t get Ruby. #2018-06-28 21:24:11 dominicm Oh really? Oh I was not expecting that response. I suppose you're not from the ruby shop that's associated with Clojure 😛. You strike me as a perl guy? 😉 #2018-06-28 21:24:20 dominicm (I'll just keep guessing acting like I knew the whole time...) #2018-06-28 21:24:24 dottedmag I feel this is the exactly the kind of conversations that cause Codes of Conduct written which precludes snarky remarks and personal attacks. #2018-06-28 21:24:30 alexmiller I have used all of these languages profesionally #2018-06-28 21:25:07 dottedmag @dominicm The point of clojure is to just work. Anywhere. Without installing anything. #2018-06-28 21:25:19 alexmiller and I respect people that like and use all of them #2018-06-28 21:25:38 dottedmag The script itself is trivial. There is no gain from using any other language. #2018-06-28 21:25:43 dominicm @dottedmag just to be clear, I actually hold no contempt for any of these languages. I feel like I have a fairly good grasp of where each one is pretty good.
I agree. I don't actually think ruby is suited here. #2018-06-28 21:26:01 alexmiller bash is about as lingua franca as it gets these days #2018-06-28 21:26:03 hagmonk bash is far and away the lowest common denominator on Unix-ish systems, with the possible exception of sh, which on many distros just links to bash. #2018-06-28 21:26:43 hagmonk it absolutely sucks, but if your goal is maximum multi-year portability, there's no other viable alternative #2018-06-28 21:27:19 dominicm bash isn't so bad. #2018-06-28 21:27:32 alexmiller I’ve grown to almost like it #2018-06-28 21:27:32 dominicm I wrote a whole panel in bash. #2018-06-28 21:27:46 dottedmag This discussion already amounts to 30% of the size of clojure(1). #2018-06-28 21:27:56 alexmiller having done the windows port to powershell, that wasn’t bad either #2018-06-28 21:28:20 dottedmag @alexmiller Oh, there is PowerShell one? In the same repo? #2018-06-28 21:28:29 alexmiller it’s on a branch I think #2018-06-28 21:28:33 alexmiller if I pushed it #2018-06-28 21:28:35 hagmonk there's a team at work that has >100k written in bash, with strict coding guidelines and practices it can work. You can grow to like many things if you immerse yourself in their idiosyncrasies :) #2018-06-28 21:28:41 alexmiller it’s not done yet #2018-06-28 21:29:34 dottedmag Found it, thanks #2018-06-28 21:29:46 dottedmag Yeat, it beats cmd #2018-06-28 21:30:20 alexmiller fer sure #2018-06-28 21:30:29 alexmiller it’s kind of like if you took bash and applied sane engineering to it #2018-06-28 21:30:49 dottedmag And isane amount of syntax, but oh well. #2018-06-28 21:31:06 hagmonk heck, powershell does autocomplete of function arguments on the command line. It's kinda impressive #2018-06-28 21:31:28 kenny Why not use ClojureScript compiled to Node? #2018-06-28 21:31:40 dominicm @kenny because node isn't always installed! #2018-06-28 21:31:52 kenny It is trivial to install. #2018-06-28 21:32:02 dottedmag Amazing. #2018-06-28 21:32:10 hagmonk @kenny the goal is not to install anything except what is required to bootstrap clojure #2018-06-28 21:32:16 alexmiller stop trying to oppress me man #2018-06-28 21:33:29 hagmonk this coming from a man comfortable writing all day in bash :) #2018-06-28 21:33:41 alexmiller the water’s fine, come on in #2018-06-28 21:33:47 dominicm @hagmonk (and any others) I just pushed up 857bbc85c33b23b5f8e51d68eb724776564d9c16 to juxt/pack.alpha which adds skinny jar support, it spits it out in multiple forms based on the parameters. Any feedback welcome! #2018-06-28 21:50:18 dominicm @hagmonk let me know if this works for you btw. #2018-06-28 21:50:54 dominicm I don't really have a concrete use case myself yet, so I'm guessing at things a little. #2018-06-28 21:57:46 hagmonk yeah for sure, I'll poke at it when I get a chance! #2018-06-29 15:18:35 dominicm I had to do a little bump, having : in filenames doesn't go well with classpaths, 800afc066907e31fe90ce4df4f519e6224116867 is the latest commit #2018-07-03 15:21:44 dominicm @hagmonk Did you get any time to see if this looks good to you? 😄 #2018-07-03 23:50:52 hagmonk Sorry @dominicm, haven't forgotten, but I've been crushed at work lately :/ #2018-07-04 05:29:23 dominicm Not at all. Entirely when you get some spare time. 😊 #2018-06-28 21:34:08 dominicm It's in alpha-alpha, as it's not in master yet, I'll be rebasing at some point, but I won't force push over those commits. #2018-06-28 21:34:13 hagmonk I can't, I didn't quote that string properly and so I dived into an empty pool #2018-06-28 21:35:01 dottedmag @hagmonk rm -rf "${PREFIX}/${SHAREDIR}". Guess what happened? True story. #2018-06-28 21:35:35 kenny Make Node required to bootstrap Clojure. #2018-06-28 21:41:40 gfredericks on the off-chance that you're not trolling, that would be way too much friction #2018-06-28 21:35:37 dottedmag @hagmonk Postinstall script in a Linux package. Root privileges. #2018-06-28 21:36:29 hagmonk Truly a trap for young players, right there … a mistake you never forget :) #2018-06-28 21:44:06 hagmonk So quick story time about the clojure installer being written in bash: the other day I was mucking with Homebridge on a raspberry pi, and needed to do some interrogation of a web API. In five minutes I had a Clojure REPL up with rebel readline, it was great. Had the installer required ruby, python, perl, etc, I would have had to dive into the Pi's package management and pull a ton of unrelated crap down, just to get a REPL. Had clojure needed a version of nodejs that was in any way incompatible with homebridge, I would have immediately faced the problem of juggling multiple npm versions. An example of the leverage you get from picking the unsexy, boring, lowest common denominator for bootstrapping tools #2018-06-28 21:44:48 dottedmag Does PowerShell come preinstalled on Windows nowadays? #2018-06-28 21:44:59 hagmonk I do believe so #2018-06-28 21:46:33 alexmiller yeah #2018-06-29 10:48:36 cfleming Of particular interest is the git lib attached to library (“Deps: org.clojure/tools.gitlibs:cab6ab”) #2018-06-29 10:49:05 dominicm Nice 🙂 #2018-06-29 10:50:13 cfleming There’s still lots of trickiness to work out, but the first 90% is done. Now there’s just the remaining 90%. #2018-06-29 11:06:40 Andreas Liljeqvist Looking forward to it, my temporary solution isn't exactly scalable... #2018-06-29 13:49:28 pesterhazy @cfleming this will be a godsend for our team #2018-06-29 14:59:24 mfikes Wow! This will be extremely useful. #2018-06-29 22:08:21 cfleming @pesterhazy Great! I know a lot of people are waiting on it. #2018-06-30 03:46:58 dorab If I try lein ancient after incorporating lein-tools-deps in my project.clj it always tells me that the artifacts are up-to-date. What do I need to do to get lein ancient work with lein-tools-deps? Thanks. #2018-06-30 05:15:32 hagmonk @dorab because with lein-tools-deps you're storing your deps in deps.edn, I'd recommend you use a tool that targets deps.edn directly. So far I only know of https://github.com/Olical/depot, but it works great #2018-06-30 15:41:30 dorab Thanks! #2018-06-30 15:48:12 dorab Did exactly what I wanted. #2018-06-30 18:04:41 stathissideris how alpha is add-lib? It fails for me like:
eggshell.controller> (deps/add-lib 'spec-provider {:mvn/version "0.4.14"})
true
eggshell.controller> (require 'spec-provider.provider)
FileNotFoundException Could not locate spec_provider/provider__init.class or spec_provider/provider.clj on classpath. Please check that namespaces with dashes use underscores in the Clojure file name. clojure.lang.RT.load (RT.java:463)
#2018-06-30 18:05:40 stathissideris If I try to add-lib again, it returns false #2018-06-30 18:05:53 stathissideris which means that it didn’t have to add it again #2018-06-30 18:11:26 dominicm @stathissideris don't know why that's happening, buut, you need to be under a DynamicClassLoader for it to work. #2018-06-30 18:11:59 stathissideris I’m not even sure what that means 🙂 #2018-06-30 18:27:04 stathissideris I’m in a REPL, is that ok? #2018-06-30 18:27:59 hiredman add-lib is very alpha #2018-06-30 18:29:44 stathissideris @hiredman sounds like I have to wait for a bit longer before I integrate it into my app #2018-06-30 20:50:39 dominicm @stathissideris it is possible to change the class loader of a repl, I believe it's in the recent history. #2018-06-30 20:53:34 alexmiller add-lib works by finding the highest dynamic class loader in the current classloader chain and altering its loader sources #2018-06-30 20:54:06 alexmiller Some repl contexts do not have a shared dcl above all repl invocations #2018-06-30 20:54:53 alexmiller So changing it would not necessarily have any effect, I think nrepl creates a new context on every evaluation #2018-06-30 20:55:50 alexmiller Using clj repl works for me but nrepl things I’ve tried do not #2018-06-30 20:56:50 dominicm we've been talking about how to fix this in nrepl. #2018-06-30 20:57:15 alexmiller Just need a dcl shared by all evals #2018-06-30 20:58:07 stathissideris @alexmiller I’m trying to integrate add-lib to my Swing app (and yes, I do realize it’s 2018 😄 ), the REPL entered the discussion because I naively thought that calling add-lib from the app would also make it available to the REPL I’m developing with #2018-06-30 20:58:33 alexmiller Are you using nrepl? #2018-06-30 20:58:39 stathissideris yes #2018-06-30 20:59:28 stathissideris but having the new libs available in the repl is not essential to me #2018-06-30 21:02:20 stathissideris I may need to read up on classloaders, I call add-lib from within a future, would that make a difference? #2018-06-30 21:21:03 stathissideris @alexmiller this is the classloader chain within the future:
(#object[sun.misc.Launcher$AppClassLoader 0x42a57993 "
Starting with (.getContextClassLoader (Thread/currentThread)). So no DCL there, I guess that explains it #2018-06-30 21:22:35 dominicm You can do a set @stathissideris #2018-06-30 21:23:16 stathissideris hm? #2018-06-30 21:23:59 stathissideris oh, setContextClassLoader! #2018-06-30 21:27:19 stathissideris @mfikes thanks, that looks like it may work 🙂 #2018-06-30 21:37:54 stathissideris @mfikes @dominicm some progress! it seems that setting the classloader as @mfikes demonstrated allows add-lib to work, but the added lib is still not available to the part of the app that needs to use it, presumably because it doesn’t share the same DCL… but now at least I understand how it works and I may be able to fix it somehow #2018-06-30 21:38:18 stathissideris many thanks to all #2018-06-30 21:39:17 mfikes @stathissideris Cool. IIRC, in the change linked above, I shoved that DCL in at the earliest possible moment. #2018-06-30 21:40:16 mfikes (It is the very first bit of code executed in main, and my hope was that any threads spawned afterward would inherit it.) #2018-06-30 21:42:58 alexmiller that should be the case #2018-06-30 23:58:15 cfleming Just to check my understanding of this - is it the case that alembic no longer works under nREPL, because of the dynapath change for Java 9? Sounds like nREPL used to actually modify the URLClassLoader but can no longer do so? #2018-07-01 05:04:47 cfleming Actually, I’ll ask this over in #nrepl #2018-07-01 03:11:43 lincpa @stathissideris I encountered a similar failure during the use of Lein::
1. Add missing dependencies in project.clj (prompt in error message) 2.lein clean #2018-07-02 00:39:32 gfredericks the +6 can be made less hacky using some sort of fancier "read up until you encounter this magic string" approach #2018-07-02 00:41:37 gfredericks startup is like 7sec though O_O
I guess -Sdeps subverts caching? #2018-07-02 12:56:52 alexmiller It used to but I think fixed in the current release #2018-07-02 15:41:23 gfredericks oooh #2018-07-02 15:41:25 gfredericks faaancy #2018-07-02 15:41:27 gfredericks I will try it #2018-07-02 15:43:32 gfredericks yep, seems better #2018-07-02 20:18:07 martinklepsch I’m running a script with clojure and have a println at the end. After that line is printed the script takes another 30+ seconds to exit. I’m probably just doing something weird but anyone an idea how to debug this? #2018-07-02 20:19:48 manutter51 I don’t know about debugging, but that looks like the kind of problem that is often solved by adding a call to shutdown-agents somewhere at the end. #2018-07-02 20:24:26 martinklepsch @manutter51 that indeed resolved the issue, thanks! #2018-07-03 08:31:31 pesterhazy @martinklepsch probably not related to the println itself, right? #2018-07-03 08:33:44 martinklepsch No, println was just literal string, but the shot down agents thing fixed it for whatever reason :man-shrugging: #2018-07-04 00:47:31 hagmonk you can try hitting it with jstack next time to see where it's hung up #2018-07-13 13:21:14 martinklepsch @U0CMVHBL2 thanks that was most likely the issue. Is there any way to properly shutdown an individual future so that I don’t have to clean up using shutdown-agents later? #2018-07-13 14:56:46 martinklepsch This issue came up pretty frequently again, tried jstack now which returns the follow (besides lots of other stuff):
"Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007f89a7016800 nid=0x5103 in Object.wait() [0x000070000957b000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c000e778> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000006c000e778> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
"Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007f89a7013800 nid=0x2e03 in Object.wait() [0x0000700009478000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c000e948> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x00000006c000e948> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
#2018-07-13 15:01:24 martinklepsch duh, found it 🙂 didn’t stop a scheduler facepalm #2018-07-13 15:04:43 martinklepsch Seems shutdown-agents is still required in some capacity though #2018-07-05 22:03:33 richiardiandrea is there a known issue in clojure exit codes not forwarded with -m ? #2018-07-05 23:01:33 ghadi I haven't seen one @richiardiandrea can you make a reproduction case? #2018-07-05 23:01:57 richiardiandrea before that I'll make sure it is not on my side 😄 #2018-07-06 11:48:28 gfredericks does customizing a repl consist of having an alternate (different from clj) executable for starting the repl, or is there a facility in tools.deps to customize it? e.g., customize the printing function, or ensure that certain code is run before the repl starts #2018-07-06 12:09:54 alexmiller Once you have a repl, you can start a nested repl within it with whatever customizations you want. But I think this an area ripe for better options. #2018-07-06 12:26:46 gfredericks looks like I can't put a system-global user.clj file on the classpath automatically, is that correct? #2018-07-06 12:27:14 gfredericks (due to "only the last :paths is kept and others are dropped") #2018-07-06 14:13:24 ghadi you can certainly start a different main method rather than clojure.main and give yourself a souped up repl @gfredericks #2018-07-06 14:34:23 gfredericks @ghadi but that requires passing args each time, right? #2018-07-06 14:43:13 ghadi si #2018-07-06 14:43:33 bronsa si`* :P #2018-07-06 14:47:42 ghadi alias repl='java $(cat jopts) -cp $(clojure -Spath) clojure.main -m gfredericks.fave.repl' #2018-07-07 07:13:13 cfleming As I understand it, aliases are never transitive. So this means that when you depend on a project using, say, :local/root then you will only ever see deps/paths from your dependency which appear in the top level of the deps.edn file, correct? #2018-07-07 07:14:20 dominicm @cfleming No transitive aliases, correct. #2018-07-07 21:58:45 martinklepsch Hey folks 👋 i have two questions: a) (how) does tools.deps support maven scopes? & b) is there a way to include dependencies with “provided” scope in the classpath? Thanks! 🙏 #2018-07-07 22:15:46 alexmiller a) not supported and b) that’s not a thing you should do anyways #2018-07-08 10:26:40 cfleming b) is absolutely useful. I use it to compile Clojure code against the IntelliJ classes, but not have them included when I run. #2018-07-08 10:27:26 cfleming What would be the deps solution to that - two separate classpaths using aliases? I guess I’d need an alias for the IntelliJ classes and would have to use that when compiling but not running? #2018-07-08 10:53:09 dominicm aliases is how I've solved it up until now. #2018-07-08 15:22:43 alexmiller Yes #2018-07-08 15:24:57 alexmiller You seem to be describing using provided scope aliases in your own project (which is useful, but handled by aliases). The question I read above though sounded like how to include transitive provided-scope deps, which does not seem like a thing you should do #2018-07-08 16:38:21 martinklepsch Yeah, I need b) for a similar tooling related concern #2018-07-08 16:53:45 martinklepsch I want to load code from Artifact A which has a “provided” dependency B. Now I’d like to make sure all these dependencies are loaded without manually listing those “provided” deps. Hope that makes sense #2018-07-08 16:56:04 alexmiller Well, the whole point of provided scope is: you provide them #2018-07-08 16:57:17 alexmiller So, not going to make something to do this #2018-07-08 17:31:19 dorab Is resolve-deps supposed to download the relevant dependencies if I call it myself? #2018-07-08 17:31:50 dorab {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "02666d70636042666d7063602f4d72766b526e677a2f3b323032"}, :content ("[email protected]")} #2018-07-08 17:36:36 dorab If it is supposed to work, what am I doing wrong above? #2018-07-08 17:37:34 dorab BTW, this is in a directory (my home dir) that does not have a deps.edn in it. #2018-07-08 21:24:17 alexmiller You need to provide the :mvn/repos to look in (these are in the install deps.edn with clj) #2018-07-08 21:53:18 dorab Ah. I thought those would automatically be included. So, I should create the full deps.edn map myself and then pass that to resolve-deps. Thanks. #2018-07-09 01:40:48 dnolen seems like there’s a bug with git deps where top-level :paths applies instead of the defaults or whatever is supplied in the git deps.edn #2018-07-09 03:22:40 alexmiller Yes, there is a ticket for it #2018-07-09 11:07:37 dnolen ah k #2018-07-09 13:04:16 dominicm Worse than the silently ignoring problem, I seem to have encountered a case where all aliases are ignored if one of them is not present. #2018-07-09 13:06:31 alexmiller I’m planning to look at this today #2018-07-09 13:12:36 dominicm fwiw, they weren't all ignored. Just no fetching took place. I haven't looked hard at this, but it confused me. #2018-07-09 20:40:59 kenny I seem to be running into an issue with local deps. If I have this directory structure
app
lib1
lib2
with app/deps.edn as
{:deps {lib2 {:local/root "lib2"}}}
and lib1/deps.edn as
{:deps {}}
and lib2/deps.edn as
{:deps {lib1 {:local/root "../lib1"}}}
and I run clj -Spath in the app directory, I get this message:
Error building classpath. Manifest type not detected when finding deps for lib2/lib2 in coordinate #:local{:root "lib2"}
Are nested local dependencies supposed to work? #2018-07-09 22:24:50 alexmiller Shouldn’t your app one have ../lib2 ? #2018-07-09 22:47:20 kenny You're absolutely right. That's what I get for working while hungry... #2018-07-09 22:51:07 hagmonk Oh wow, I came here to report the same thing, except I've had lunch so I know it's a bug :) #2018-07-09 22:53:19 kenny I think I'm hitting the same problem with a slightly different directory structure. Except this time it is correct.
lib1
lib2
samples/sample1
with lib1/deps.edn as
{:deps {}}
and lib2/deps.edn as
{:deps {lib1 {:local/root "../lib1"}}}
and samples/sample1/deps.edn as
{:deps {lib2 {:local/root "../../lib2"}}}
and I run clj -Spath in the samples/sample1 directory, I get this message:
Error building classpath. Manifest type not detected when finding deps for lib1/lib1 in coordinate #:local{:root "../lib1"}
#2018-07-09 22:53:46 hagmonk Yeah same issue:
./libs/dep-b
{:deps {org.clojure/clojure {:mvn/version "1.9.0"}}}
./libs/dep-a
{:deps {dep-b {:local/root "../dep-b"}}}
./apps/foo-app
{:deps {dep-a {:local/root "../../libs/dep-a"}}}
#2018-07-09 22:55:35 hagmonk I can work around it by having libs/dep-a refer to ../../libs/dep-b instead, but that is only a coincidence in this case, if apps was one level deeper it wouldn't work #2018-07-09 22:59:04 kenny Oh ok. That's a nasty one. #2018-07-09 22:59:32 hagmonk push the "vote" button :) #2018-07-09 23:03:08 kenny It looks like the user in that issue is able to get output from clj -Spath though. #2018-07-09 23:03:46 kenny But it sounds identical. Perhaps an older version of tools-deps didn't have this error message. #2018-07-09 23:07:45 hagmonk I thought :deps/root might be another way to dig oneself out of this hole, but perhaps I'm misunderstanding how that works #2018-07-10 12:48:56 pesterhazy I know that you can specify local exclusion in deps.edn like so "reagent {:mvn/version "0.7.0", :exclusions [cljsjs/react-dom cljsjs/react cljsjs/create-react-class cljsjs/react-dom-server]}" Is there a way to specify global exclusions as well? #2018-07-10 14:41:29 alexmiller No #2018-07-12 04:18:08 cfleming I’m working on some tools.deps testing, and I can’t figure this out. I’m calling resolve-deps like this:
(resolve-deps
{:deps {'local {:local/root "/Users/colin/dev/tools-deps-test/library"}}}
nil)
#2018-07-12 04:18:47 cfleming That deps.edn looks like this:
~/d/cursive (deps)> cat /Users/colin/dev/tools-deps-test/library/deps.edn
{:deps {org.clojure/tools.gitlibs {:git/url ""
:sha "cab6abf0122cffb9e6b947c838458c69a0992031"}}}
#2018-07-12 04:19:17 cfleming What I’m getting back in my lib map includes this:
local {:local/root "/Users/colin/dev/tools-deps-test/library",
:deps/manifest :deps,
:deps/root "/Users/colin/dev/tools-deps-test/library",
:paths []},
#2018-07-12 04:19:31 cfleming I can’t figure out why the :paths vector is empty here. #2018-07-12 04:19:58 cfleming /Users/colin/dev/tools-deps-test/library/src exists. #2018-07-12 04:41:42 cfleming Ugh, never mind me, I’m not merging in my config maps. #2018-07-12 04:41:57 cfleming This works:
(let [deps (reader/merge-deps [(reader/slurp-deps "/usr/local/Cellar/clojure/1.9.0.381/deps.edn")
(reader/slurp-deps "/Users/colin/.clojure/deps.edn")
(-> {:deps {'local {:local/root "/Users/colin/dev/tools-deps-test/library"
:deps/manifest :deps}}}
(#'reader/canonicalize-all-syms))])]
(resolve-deps deps nil))
#2018-07-12 15:38:46 kenny It seems tools-deps suffixes all Git deps with each path in my :paths vector. If I have ["src" "foo1" "foo2"] for my :paths, the output of clj -Spath will suffix each Git dependency with each path. i.e. /Users/kenny/.gitlibs/libs/lib1/lib1/f31e19ffe371a60f0474759451dba9b88bf7ef04/src:/Users/kenny/.gitlibs/libs/lib1/lib1/f31e19ffe371a60f0474759451dba9b88bf7ef04/foo1:/Users/kenny/.gitlibs/libs/lib1/lib1/f31e19ffe371a60f0474759451dba9b88bf7ef04/foo2. Is this intentional? #2018-07-12 15:39:13 dominicm known bug #2018-07-12 15:40:27 kenny Is there an issue? #2018-07-12 15:42:03 dominicm There is, although I'm not sure of the ref for it #2018-07-12 19:24:16 avi I got bit by that yesterday, working with a local lib. My app has main test-runner etc under src but the lib doesn’t have main so none of the clj files could be found by require/load #2018-07-12 19:25:04 avi Ah no, my problem was slightly different #2018-07-12 19:25:08 avi Almost the inverse #2018-07-12 19:25:20 avi (Just read the issue more closely) #2018-07-12 19:25:35 avi I’ll try to check whether an issue for my experience already exists. #2018-07-13 02:22:49 dnolen are :extra-deps actually merged? like if in an alias I want :exclusions but not otherwise? #2018-07-13 02:32:17 alexmiller I’m not sure I understand what you’re asking but it sounds like :override-deps #2018-07-13 02:33:32 dnolen oh! cool 🙂 #2018-07-13 02:35:03 dnolen thanks #2018-07-13 12:21:36 martinklepsch Not sure if a bug but clj -Sdeps '{:deps {org.asciidoctor/asciidoctorj {:mvn/version "1.5.6"}}}' -Spath doesn’t pull in the dependencies below (lein/boot do)
[org.asciidoctor/asciidoctorj "1.5.6"]
├── [com.beust/jcommander "1.35" :scope "runtime"]
└── [org.jruby/jruby-complete "1.7.26" :scope "runtime"]
#2018-07-13 12:26:01 dominicm Runtime scope dependencies aren't or weren't brought in. I think they are now. #2018-07-16 11:45:53 martinklepsch Do I need to manually upgrade tools-deps for that to take effect somehow? #2018-07-16 11:45:57 martinklepsch sorry noob q 😄 #2018-07-16 12:10:39 dominicm I guess you bump your version? Not sure when the last release was #2018-07-13 21:35:18 dnolen I guess there’s no error on non-existent alias? #2018-07-13 21:36:19 alexmiller Ha, I just committed that within the last hour :) #2018-07-13 21:36:28 alexmiller Will be in next release #2018-07-13 22:23:40 rberger If one’s ssh key for github is not id_rsa or any of the default ssh names, do you need to specify the name somewhere in deps.edn? I can git clone a private dependency repo with my keys in the ssh-agent, but the clojure command always ends up with: #2018-07-13 22:23:43 rberger Error building classpath.
#2018-07-13 22:41:03 rberger This is on macos High Sierra and Clojure 1.9.0.381 #2018-07-13 23:29:09 ghadi This is an awful answer, and it's a bit of a mess, and I'd really like to fix it once and for all by shelling out to git directly (golang does this):
@rberger
for now, make sure the environment SSH_AUTH_SOCK is set to your ssh-agent's socket path
move any extraneous Identity keys in your .ssh/config out of the way #2018-07-13 23:29:21 ghadi JGit / JSch tries to read them and gets confused #2018-07-13 23:29:39 ghadi (I use ed25519 keys, which are common these days...only supported via the ssh agent) #2018-07-13 23:39:28 rberger Thanks! It looks like I had some things in the ~/.ssh/config that was causing the problem. Might be good to at least mention this in the docs (https://clojure.org/reference/deps_and_cli) #2018-07-13 23:41:52 rberger Though the info in the config file is the correct path to the IdentifyFile.
Host
User git
Port 22
HostName
IdentityFile ~/.ssh/github.rsa
AddKeysToAgent yes
TCPKeepAlive yes
IdentitiesOnly yes
#2018-07-15 14:01:04 alvina there is command line tool for finding latest version of the software #2018-07-15 14:01:22 alvina can some body remind me the name of that command? #2018-07-16 00:37:57 alvina thanks a lot #2018-07-16 08:14:11 levitanong Hi all, trying to get the clojure:tools-deps docker image to work with gitlab-ci. Does anyone have experience in the matter? #2018-07-16 09:16:34 Andreas Liljeqvist I have never used gitlab-ci, but here is my .yml for running on bitbucket.
I imagine it would look quite similar.
https://pastebin.com/SE1shJE3 #2018-07-16 10:42:29 delaguardo What kind of problems you are seeing? #2018-07-16 11:07:49 levitanong @U04V4KLKC clj: command not found. Same with clojure. I’ve tried purposefully misspelling the docker image to see if there’s a distinct error when the docker image isn’t found, but it appears gitlab-ci just silently fails, and proceeds to run as if from a blank image. #2018-07-16 11:08:23 levitanong @U7YG6TEKW Yeah, mine looks similar, except gitlab complains that the clojure command isn’t found. #2018-07-16 11:15:38 delaguardo clj should is not available cause readline is not installed. But clojure should be there. #2018-07-16 11:16:34 delaguardo Could you show your config file? #2018-07-16 11:21:03 delaguardo Also I can suggest to use clojure:tools-deps as your image so it is hard to misspell the name #2018-07-16 11:28:52 levitanong test:tools-deps:
image: clojure:tools-deps-alpine
script:
- clojure -A:test
#2018-07-16 11:29:09 levitanong Skipping Git submodules setup
$ clojure -A:test
bash: line 52: clojure: command not found
ERROR: Job failed: exit status 1
#2018-07-16 11:33:44 delaguardo That is odd… My config file looks a bit different, but it should not be a problem #2018-07-16 11:34:41 levitanong One of the first things I tried was:
image: clojure:tools-deps
test:
script:
- clojure -A:test
#2018-07-16 11:34:49 levitanong same error 😮 #2018-07-16 11:37:33 delaguardo Which version of gitlab-runner you are using? #2018-07-16 11:38:24 levitanong Running with gitlab-ci-multi-runner 9.5.1 (96b34cc) #2018-07-16 11:41:36 delaguardo I’m using 11.0. Let me downgrade, it can be a problem #2018-07-16 11:43:06 levitanong ooh #2018-07-16 12:00:21 delaguardo Unfortunately, I can not downgrade to 9.5.1 on my mac. Looks like a bunch of dependencies conflicts there. Is it possible for you to upgrade yours and try again? #2018-07-16 12:02:24 levitanong Hmm. I’m working off of my company’s on-prem installation of gitlab. I’ll ask the dev-ops guy if he can upgrade it. I’ll keep you posted! #2018-07-16 14:12:47 levitanong Apparently our devops guy needs to backup all our repos over the weekend to upgrade gitlab. I’ll prolly have to get back to you in a week. Good thing lein still works! #2018-07-24 03:43:08 levitanong hello, tried again on gitlab-runner 11.1.0. Still says clojure: command not found. Do you think it’s because we’re using the on-prem community version of gitlab? #2018-07-24 06:28:06 levitanong Oop, scratch that. we had a faulty config somewhere else. Works now, thanks! #2018-07-24 06:44:39 delaguardo Nice! parrot #2018-07-16 09:58:54 darnok Hi. I have a problem w tools-deps. I'm using :local/root quite extensively and I noticed a problem. If I have a project project1 that points to another library with {:local/root "../../libs/abc"} and then that library uses another library {:local/root "../other-lib" it doesn't resolve the abc library classpath correctly. #2018-07-16 10:00:01 darnok It resolves the second local/root according to the project1 root. #2018-07-16 10:44:56 dominicm yeah. There is a JIRA for that. #2018-07-16 10:45:19 dominicm I've been able to "flatten" everything up until now though. #2018-07-16 11:45:08 martinklepsch # clj
Please install rlwrap for command editing or use "clojure" instead.
# echo $?
0
has changing the status code here to some non-zero value been discussed before? #2018-07-16 12:08:36 delaguardo I remember such discussion but can’t find an issue. But I’m definitely want to vote for it #2018-07-16 12:44:20 alexmiller no ticket, but feel free to log on and/or provide a patch #2018-07-16 13:44:47 martinklepsch @U064X3EF3 can you point me to the source of the clj clojure scripts maybe? thanks 🙂 #2018-07-16 12:04:43 stathissideris hello, is there some special magic to make git deps from private repos work? I’m using the {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "c8afa1bc88afa1bca0bdaae6aba7a5"}, :content ("[email protected]")} URL and auth fails #2018-07-16 12:07:14 stathissideris thanks! I’ll have a look #2018-07-17 13:16:16 martinklepsch Is there a way to specify an entirely different set of :deps for an alias? (i.e. no merge, no override, just different value) #2018-07-17 13:21:37 dominicm @martinklepsch in what context? #2018-07-17 13:23:58 martinklepsch @dominicm not sure I understand what you mean by context? 😄 #2018-07-17 13:32:03 martinklepsch I’d like to swap out the value of :deps in the top level map by another value via an alias #2018-07-17 13:54:31 dominicm So from the clj command line use? What's the use case? #2018-07-17 14:11:58 ghadi :override-deps should do it, but I think there is an acknowledged bug when the override is of a different "type" (mvn/git) #2018-07-17 14:58:42 dominicm @ghadi I think @martinklepsch means that he wants to have:
{:deps {A {:mvn/version "1.1.0"}}
:aliases {:B {:the-new-deps {B {:mvn/version "5.4.1"}}}}}
And when doing clj -A:B that should result in only B being on the classpath #2018-07-17 14:59:34 ghadi oh #2018-07-17 14:59:51 martinklepsch yeah, that’s it @dominicm #2018-07-17 15:00:18 ghadi I think this has come up before -- not supported by aliases, but perhaps by a different deps.edn file #2018-07-17 15:00:23 dominicm I don't understand why you'd practically need that though 🙂 #2018-07-17 15:00:35 dominicm I suppose you could actually have:
{:deps {}
:aliases {:A {:extra-deps {A …}} :B {:extra-deps {B …}}}}
#2018-07-17 15:01:57 martinklepsch yeah aliases would work I think #2018-07-17 15:02:30 martinklepsch I’m not 100% sure I need it either but I come up with some approaches and have no idea what tools-deps allows for so just asking 😄 sorry for the noise 🙂 #2018-07-17 15:13:29 ghadi no problem @martinklepsch not noise at all #2018-07-17 15:13:49 ghadi It does seem a bit "off-label prescription" #2018-07-17 15:13:55 ghadi (the use case) #2018-07-17 15:31:49 martinklepsch @ghadi cool, thanks 🙂 #2018-07-17 15:32:15 martinklepsch If I specify a deps.edn with -Sdeps any local deps.edn in the working directory should be ignored — is that correct? #2018-07-17 15:32:51 dominicm No, though others have tripped over that wording too. #2018-07-17 15:33:58 martinklepsch uh, yeah, especially that “final” part: “Deps data to use as the final deps file” #2018-07-17 15:37:16 martinklepsch is there another way to achieve it then? specifying a file would also be fine, just looking for something that ignores an existing deps.edn #2018-07-17 16:53:00 dominicm Final in the context of merging together deps.edn files 🙂 #2018-07-17 17:45:55 martinklepsch Hm, the word “final” creates a different expectation for me 😄
So is the only way to run a clj process with a deps.edn that’s different than the one in the current working directory to move files around? #2018-07-17 18:01:46 dominicm I think the proposal was to change it to last. #2018-07-17 18:02:44 dominicm Not that I know of. I think pack is unique in utilizing tools.deps.alpha in that way, and I'm not sure it really works due to the cwd being internal to t.d.a rather than passed in. #2018-07-17 18:11:48 martinklepsch Maybe another option could be added #2018-07-17 18:57:05 alexmiller -Sdeps! #2018-07-17 19:23:40 dominicm How well does that work in shells? #2018-07-17 22:38:11 alexmiller I was actually kidding :) #2018-07-18 05:14:17 dominicm Vim wasn't (for its internal commands). #2018-07-18 08:17:00 dominicm Oh actually, when add-lib refers to the "highest" class loader, I guess that means it would find the parent one if I set it, so I could fix this in nrepl #2018-07-18 08:40:05 martinklepsch Under what conditions is it expected that a user has to manually delete .cpcache? #2018-07-18 09:54:01 dominicm bugs? 😄 What happened? #2018-07-18 09:54:36 dominicm I wouldn't expect to ever have to from what I know, fixes are being made to handle cases where it isn't being updated correctly #2018-07-18 10:17:21 pesterhazy @U050TNB9F I think this is an interesting question #2018-07-18 12:24:11 alexmiller Generally using -Sforce to force recomputing a single classpath makes more sense. One known case where this is required is when local transitive deps change their dep structure - right now that won’t be automatically discovered #2018-07-18 13:00:22 martinklepsch > local transitive deps change their dep structure
that might have been it then #2018-07-18 14:46:44 alexmiller I have a plan to fix this, but it’s kind of involved #2018-07-18 10:22:34 martinklepsch With git-libs is there a way to specify another “root” path to look for a deps.edn in? #2018-07-18 10:22:44 martinklepsch I remember this being discussed previously but can’t recall #2018-07-18 10:22:49 dominicm Yep! I think :deps/root is the key #2018-07-18 10:51:20 dominicm do you have a deps.edn file? #2018-07-18 10:51:23 dominicm or pom.xml #2018-07-18 10:51:52 cfleming Doesn’t :deps/manifest only support :deps at the moment? #2018-07-18 10:52:09 cfleming i.e. deps only knows how to extract path info from other deps projects? #2018-07-18 10:53:56 dominicm I think there's a :deps/manifest :mvn implementation #2018-07-18 10:55:07 martinklepsch duh, I thought I commited the deps.edn file but didn’t 🙈 #2018-07-18 22:49:55 zentrope I’ve got a github repo with mname/foo with a clojure project inside it, mname/foo/clojure-project/: is it possible to use that for a deps.edn git dependency? #2018-07-18 22:50:27 zentrope Answer seems to be “no” as I can’t reference a github URL + subdir. #2018-07-18 22:51:39 zentrope Oh, is that the :deps/root thing I see up above there? Hm. #2018-07-18 22:55:07 zentrope Getting a “don’t know how to create ISeq” from the -m namespace part of the CLI, but at least it seems to find the right subdirectory. #2018-07-18 22:55:41 zentrope Oops, typo. Nevermind. #2018-07-18 22:55:55 zentrope :+1: #2018-07-19 03:25:09 cfleming Still lots to do and lots of potential changes. #2018-07-19 08:32:58 dominicm @claudiu I'd be interested in adding uberwar support to https://github.com/juxt/pack.alpha I'm not familiar with the format really though and what would be needed. Is it just a jar with a particular manifest? #2018-07-19 09:11:21 claudiu not really sure to be honest. 🙂 I am deploying my test-app to google-appengine, and that's the format there. #2018-07-19 09:12:22 claudiu Added lein-tools-deps to my project, seems to work for development but the uberwar seems to compile multiple times the same file, and when I run it I get some strange errors. #2018-07-19 09:19:36 dominicm I can try find some time to look into it for you. #2018-07-19 09:23:53 dominicm Looks like a jar with a specific fs layout, shouldn't be too difficult to get working with pack, I'm a little worried about git dependencies, that might need some testing. #2018-07-19 10:13:11 claudiu Cool 🙂 Could help with testing git & normal, and see how it matches the lein one. #2018-07-19 11:24:26 claudiu Having the uberwar option would be really awesome. #2018-07-19 17:28:18 dominicm Is there an easy way to test an uberwar locally? #2018-07-20 10:06:01 dominicm Thanks, I'll take a look #2018-07-19 15:44:40 alexmiller released tools.deps.alpha 0.5.442, clj 1.9.0.391 #2018-07-19 15:45:04 alexmiller - Throw error if unknown aliases are used (TDEPS-85)
- Fix bad break character in rlwrap (TDEPS-77)
- Use non-0 exit code in clj if rlwrap doesn't exist (TDEPS-86 - thanks Martin Klepsch!)
- Change wording for -Sdeps in help and man for clarity (TDEPS-87)
#2018-07-20 00:09:29 cfleming I’m testing deps with Cursive, and I’m finding that resolving several large projects at once is really slow. This is a project with several apps in it, all of which use a series of libraries transitively via ‘:local/root’. Is it possible that each project refresh works out a complete list of what to download up front and then downloads it, such that each project will download everything even if another project has concurrently downloaded that dep in the meantime? #2018-07-20 00:14:41 cfleming In total there are 15 projects (including apps + libs), it uses kafka so who knows how much each one will download. But I’m on a 1 Gb fibre connection, so it’s generally pretty quick. It’s taking around 15-20 minutes to resolve them all. #2018-07-20 00:32:57 alexmiller It uses Maven to download as it goes, which uses the Maven cache #2018-07-20 00:33:42 alexmiller That said, I have spent no time optimizing any of that and none of it is parallel (but most likely some could be) #2018-07-20 00:34:41 cfleming Right, my case is probably unusual in that if you set up a large project in Cursive (which I just did based on data a user sent me) then IntelliJ will go off and resolve all the projects in parallel. #2018-07-20 00:40:52 cfleming Does the maven lib separate the resolution into a resolution step and a download step? I’m wondering if each project does the resolution step, decides what to download and then downloads it without checking before each download if something else has done so in the meantime? #2018-07-20 12:59:33 mping Hi, is there a recommended way to integrate lein with tools-deps? I want to use git dependencies #2018-07-20 13:09:37 mgrbyte @mping Not sure about "recommended", but I'm using lein-tools-deps - works pretty well so far. Only things that don't work are lein deps :tree, lein search AFAICT #2018-07-20 13:09:51 mping @mgrbyte :+1: #2018-07-20 15:31:46 hlolli is there a function in tools.deps that I could use to generate a vector of maps containing all remote dependencies for a project's dependencies, plus, if possible, the sha-sums of the .jar files. I'm wiring a deriviation for nix(Os), and it seems that there's no tool for generating nix expression for clojure dependencies available. There's one for maven, called mvn2nix. I guess I could use that, but I'd have to make some hack for mirrors, adding clojars etc.. #2018-07-20 15:32:55 dominicm yeah, I think that would be possible via the api #2018-07-20 15:33:09 dominicm the api is small, call both of the documented functions #2018-07-20 15:33:59 hlolli make-classpath and resolve-deps? I will try! #2018-07-20 15:35:40 dominicm maybe not make-classpath actually #2018-07-20 15:35:45 dominicm just resolve-deps & vals #2018-07-20 16:49:41 hlolli @dominicm thanks for the tip, I'm getting all the deps printed, which is good. But that which is missing, is the url's to the .jar files
fipp
{:versions
{#:mvn{:version "0.6.12"}
{:mvn/version "0.6.12", :deps/manifest :mvn}},
:paths {#:mvn{:version "0.6.12"} #{[]}},
:select #:mvn{:version "0.6.12"},
:pin true},
probably straightforward to do, but how would I get the download urls from this information? #2018-07-20 16:51:15 dominicm @hlolli I think there's something missing here, you've called resolve-deps? #2018-07-20 16:51:38 hlolli yes I did this
(prn
(pprint (resolve-deps '{:deps
{org.clojure/clojure {:mvn/version "1.9.0"}
org.clojure/clojurescript {:mvn/version "1.10.238"}
org.clojure/tools.reader {:mvn/version "1.3.0-alpha3"}
com.cognitect/transit-cljs {:mvn/version "0.8.256"}
malabarba/lazy-map {:mvn/version "1.3"}
fipp {:mvn/version "0.6.12"}}}
{:verbose true}
)))
#2018-07-20 16:55:30 dominicm Something looks off about your result to me. The value of :paths for me is different, I have jars in there. #2018-07-20 16:55:32 dominicm or dirs, context dependent. #2018-07-20 16:57:16 hlolli ah yes, if the jar is found, it appears #2018-07-20 16:57:21 hlolli but only locally #2018-07-20 16:57:46 hlolli com.google.protobuf/protobuf-java
{:mvn/version "3.0.2",
:deps/manifest :mvn,
:paths
["/home/hlolli/.m2/repository/com/google/protobuf/protobuf-java/3.0.2/protobuf-java-3.0.2.jar"],
:dependents [com.google.javascript/closure-compiler-unshaded]},
#2018-07-20 17:01:11 dominicm You want the remote values? #2018-07-20 17:03:34 hlolli yes, I need that, and the sha sums, but I could just get them after all is downloaded #2018-07-20 17:07:46 hlolli the pain with nix, is that nothing can be downloaded without being declared, all http fetches need to be done on nix's side #2018-07-20 17:08:34 dominicm I don't think you can get the remote urls, I think it's blackboxed into a maven library #2018-07-20 17:09:39 hlolli maybe, you can always download jars directly from clojars, maybe maven is more blackboxed #2018-07-20 17:10:01 dominicm I meant maven library which handles clojars & maven #2018-07-20 17:10:11 dominicm t.d.a has no concept of "get-url-for-dep" #2018-07-20 17:10:43 dominicm You'd probably have to utilize the same approach that mvn2nix uses, and also write equivalents for git & local deps. #2018-07-20 17:12:06 hlolli pff, hope its easy 🙂 will research. #2018-07-20 22:31:19 noprompt has anyone run in to problems with using krypton and git deps? #2018-07-20 22:32:03 noprompt clj complains it can’t find the ~/.ssh/id_krypton private key which it shouldn’t because, as i understand it, krypton has that key. #2018-07-21 05:32:20 dominicm @noprompt I imagine jgit doesn't support krypton #2018-07-21 09:39:53 dominicm @alexmiller @ghadi curious whether you've looked into libgit and whether it could be used in place of jgit to solve this problem? Maybe before falling back on jgit on any unsupported platforms? #2018-07-21 09:40:11 dominicm > Linux, FreeBSD, OpenBSD, Mac OS X, iOS, Amiga, MinGW and fully native Windows.
is pretty good though, "Amiga", like, come on! #2018-07-21 09:40:48 dominicm There's a library, "jagged" for doing libgit w/ java, but it's not on maven. #2018-07-21 13:32:51 pesterhazy if it's fast enough, shelling out to /usr/bin/git seems optimal #2018-07-21 13:36:45 dominicm Having tried to wrap libgit2 this morning, I'm thinking that shelling out is increasingly valuable. I'm wondering about environments which may not have the git binary available though. #2018-07-21 13:36:54 dominicm Having libgit2 embedded for that situation is very valuable. #2018-07-21 13:37:19 dominicm Of course, /usr/bin/git falling back onto jgit or something would work for that pretty well #2018-07-21 13:48:16 pesterhazy true, I didn't think of the case where the binary is not available (not unrealistic in this world of Docker) #2018-07-21 18:10:42 athomasoriginal Does anyone know of examples of clj cli tool + deps.edn monorepo setups? I have been studying Juxt's Edge repo, but this does not yet include any other modules. My assumption is that one would create a :dev alias where, instead of referencing the artifact, it references the local path to your module...curious as to how other's approach this 🤔 #2018-07-21 19:12:48 ghadi We need to shell out to git to solve all the ssh related issues #2018-07-21 19:13:12 ghadi All the other git based deps tools do that, like golangs #2018-07-21 19:24:54 athomasoriginal @ghadi sorry, was this in reference to my question? #2018-07-21 19:25:26 ghadi Sorry, earlier Q about libgit #2018-07-21 19:40:27 dominicm @ghadi libgit doesn't do the same thing as git? #2018-07-21 20:59:08 cfleming IntelliJ also shells out to Git for its support. #2018-07-21 20:59:38 cfleming There’s probably a bunch of useful support code there if we need it (Apache licensed) #2018-07-22 19:13:07 uwo are :exclusions only for mvn deps, or do they work for git deps as well? Running clj -Stree made me think they were, because the exclusion list appeared to be applied to the dep tree #2018-07-22 20:21:15 alexmiller Should work for both #2018-07-22 21:30:18 uwo awesome thanks! #2018-07-22 20:20:19 hlolli So I succeeded with using tools.deps for generating nix-expressions, https://github.com/hlolli/clj2nix now it should be easy for nix users to start bombarding clojure-based application into nixpkgs, or use nix to build/deploy clojure applications. If you're a nix user, please give it a try 🙂 #2018-07-22 21:57:49 dominicm @hlolli how did you do it? #2018-07-22 22:02:28 hlolli @dominicm after looking at a nix function fetchMaven, I realized that all maven2 remote url's follow a pattern, from there I discovered fetchMavenArtifact, which I use, for both clojar and maven central. Just try both for each dep, and both clojar and maven central work with same url format. #2018-07-22 22:03:29 dominicm @hlolli do you handle git and local? #2018-07-22 22:08:26 hlolli good question. I don't handle it automatically. And it's possible to do it. #2018-07-22 22:09:02 hlolli Another thing I need to add is classifier. Which I don't handle, or the funtcion fetchMavenArtifact doesn't handle it. #2018-07-22 22:20:09 dominicm I'm not certain if t.d.a handles classifier yet, I might be wrong #2018-07-22 22:20:58 hlolli I'm about to test it 🙂, curious. #2018-07-22 22:57:46 alexmiller It does #2018-07-22 22:59:38 alexmiller With the caveat that classifiers do not distinguish in a lib (they’re part of the coord, not the lib key), so you can’t use multiple classifiers (or no classifier) for the same lib at the same time #2018-07-22 23:00:21 alexmiller It also only handles jars and ignores other extension types #2018-07-23 10:42:29 cfleming Is there a rough ETA on Windows/Powershell support? #2018-07-23 14:21:02 alexmiller Sorry, I just haven’t had time to pull it over the line yet. I expect it will be a couple weeks before I get back to it #2018-07-23 21:14:49 gfredericks I'm thinking about how to best replace all the tools that in leiningen mostly consisted of :injections, without creating a combinatorial explosion of mains
I had the idea that maybe that could be what a single user.clj could be for #2018-07-23 21:15:21 gfredericks I'm assuming tools.deps lets you add common source paths in the user's deps.edn, but I might have made that up #2018-07-24 00:16:29 gfredericks it works, at least:
:paths ["/home/memyself/.clojure/user-classpath"] #2018-07-24 23:38:43 souenzzo real funny the kind of problem and solutions that windows users do in 2018.
WLS still not a thing? #2018-07-25 13:37:41 timgilbert WSL work pretty well, and you can run the ubuntu version of tools.deps on it with no problems. The pain comes when you are trying to interoperate with native Windows stuff - eg, IntelliJ runs natively, so it has its own notion of an m2 repository, classpath, etc #2018-07-24 16:04:07 ghadi @argument files
one or more argument files containing options
Java now accepts a file to read arguments ^ #2018-07-24 23:23:28 gfredericks the worst part is having to hard-code my home directory in the deps.edn; I'm not sure if there's a way to avoid that #2018-07-25 00:25:35 avi Relative paths? Paths that include things like ~ ? (Not sure what kind of thing ~ is. Alias?) #2018-07-25 00:42:07 gfredericks it's a bashism I believe #2018-07-25 00:42:19 gfredericks . and .. are a lot more first-class than ~ #2018-07-25 00:42:57 gfredericks I can't use ~ because it's just bash, and I can't use . and .. because I assume that'd refer to the working directory of the java process, which could be anywhere #2018-07-27 11:02:14 gfredericks but this is in a deps.edn file, where, unless I'm mistaken, there's no opportunity to execute code #2018-07-28 09:32:19 pesterhazy right, I meant as a potential feature for tools.deps. Many tools parsing yaml/json/ini configuration files process filenames with expanduser #2018-07-28 12:35:28 gfredericks ah right #2018-07-26 13:31:16 mitchelkuijpers Is anyone here using this on circleci? I think it is a good idea to cache the .cpcache folder? Because I saw this in the reference guide:
The cached classpath file is used when:
* It exists
* It is newer than all deps.edn files
I would guess if i checkout the project and then restore a cache the .cpcache folder is always newer #2018-07-26 13:31:48 mitchelkuijpers Hmm this is exactly what you want if I base the cache on deps.edn.. While I typed this I figured it out lol #2018-07-26 14:04:24 mitchelkuijpers Is there some place where it caches git dependencies? I am currently caching the .m2 folder and the .cpcache folder #2018-07-26 14:26:32 mfikes @mitchelkuijpers by default ~/.gitlibs #2018-07-26 14:28:12 mitchelkuijpers Ah nice thnx #2018-07-26 19:07:24 Peter Wilkins hi, anyone know how to add a local maven repository to a deps.edn file? #2018-07-26 19:11:17 mfikes :mvn/local-repo #2018-07-26 19:16:01 mfikes So, for example, this will create and stash stuff in ./my-local-repo
clj -Sdeps '{:mvn/local-repo "my-local-repo"}'
#2018-07-27 11:09:58 gfredericks is add-lib likely to get officialized anytime soon? #2018-07-27 14:25:09 alexmiller Define soon #2018-07-27 15:09:50 mikerod I’m seeing 2 definitions:
1) in or after a short time.
2) used to indicate one’s preference in a particular matter.
but more interesting, the origin:
Old English sōna ‘immediately.’
😛 #2018-07-27 15:58:13 alexmiller Then I’ll go with no :) #2018-07-27 21:01:10 ghadi notes on git deps with
https with authentication is not supported yet by clj (https is 65% of github's traffic)
ssh authentication via Java is a usability trashfire (some KEX methods unsupported)
The terminal is not interactive in clojure, so you cannot accept unseen public host keys - this might be ok, but surprising.
Esoteric .ssh/config options can break cloning because the underlying java ssh libraries misread the .ssh/config file
ed25519 keys supported via ssh-agent, but not via ~/.ssh/config entries
Porting to shell out to git would help all of these issues, if remains compatible with Windows support #2018-07-27 21:01:20 ghadi notes I've previously pasted here ^ #2018-07-28 12:36:50 gfredericks would support for ~ in paths in tools.deps (or parts of it? e.g. just the user file) be a welcome feature? I'd be happy to work on it #2018-07-28 12:47:05 alexmiller My first reaction is no #2018-07-28 12:47:35 alexmiller But feel free to file a jira and I’ll think about it #2018-07-28 14:02:52 gfredericks roger thanks #2018-07-28 15:26:45 gfredericks can git deps specify a subdirectory of the repo? #2018-07-28 15:39:11 gfredericks I'm making a library called user.clj and all it does is contain a user.clj file that loads other user files #2018-07-28 15:39:41 gfredericks e.g., ~/.clojure/user.clj, and it could probably search the classpath for user-<n>.clj #2018-07-28 15:39:49 gfredericks and you could disable it with env variables #2018-07-28 15:42:10 mfikes Git deps can point at a subdirectory. I can’t recall the key. Maybe :deps/root or some such #2018-07-29 15:01:56 dominicm Are they merged? Or something else? #2018-07-29 15:13:23 gfredericks well right now it only supports a single file, so that's not an issue #2018-07-29 15:14:01 gfredericks if I can think of a performant way of searching the classpath for other files, then (once defined what's in scope (`user.autoload.*`?)) I'd probably just load them in alphabetical order #2018-07-29 15:14:53 gfredericks I don't think merely saying "all user.clj files will be loaded!" is possible because AFAIK clojure just loads an arbitrary one, so there's no guarantee it would be mine #2018-07-29 15:15:05 gfredericks so the whole approach assumes that the rest of your user files are named something different #2018-07-29 15:15:34 gfredericks (note that ~/.clojure/user.clj would not, in general, be on the classpath) #2018-07-29 15:19:10 dominicm What does the library do then? I'm a bit confused! #2018-07-29 15:19:33 dominicm I thought this was a new version of ./util #2018-07-29 15:20:35 gfredericks oh no, not at all #2018-07-29 15:20:49 gfredericks > The immediate problem it's meant to solve is that the user-level deps.edn file can't reference $HOME-relative paths as a way of providing your own user.clj directly. #2018-07-29 16:39:58 dominicm I see... So this is an alternative to having an alias in your own ~/.clojure/deps.edn pointing at a dir? #2018-07-29 16:57:01 gfredericks yes, especially because doing it that way means you need an absolute path #2018-07-29 16:57:28 gfredericks so there seems to be no way to make it relative to $HOME #2018-07-30 02:37:16 cfleming Why does t.d.a have the org.clojure/clojure dep specified as provided? #2018-07-30 02:57:23 seancorfield That seems correct to me: it's always going to be run in a context that already has some version of Clojure loaded. #2018-07-30 02:58:24 seancorfield (shouldn't most Clojure projects assume Clojure is "provided"?) #2018-07-30 04:09:25 richiardiandrea The only exception IMHO is cljs self-host which would not require any dependency on clojure #2018-07-30 04:45:05 cfleming > shouldn’t most Clojure projects assume Clojure is “provided”?
Maybe, but none of them do. #2018-07-30 04:48:07 cfleming It means that if I get the classpath for t.d.a from Maven or whatever, it won’t contain Clojure, which it clearly needs. #2018-07-30 05:02:18 seancorfield Yeah, I've tried to drum up support for a movement to change the Clojure dep to "provided" in projects but... inertia... 🙂 #2018-07-30 05:20:13 cfleming Thinking about it though, none of those libs work without Clojure so it really is a dependency for them. #2018-07-30 05:31:46 seancorfield It can be argued both ways. Clojure itself is kind of a weird "dependency" because in most projects, the choice of Clojure version comes first and then all the other dependencies are added. So, in that sense, Clojure is "provided". But, yeah, you can also argue Clojure "really is a dependency" -- but it's often going to be a different version for each of the other dependencies...? #2018-07-30 05:33:22 alexmiller I think you have to separate app and lib contexts here too #2018-07-30 05:33:44 alexmiller Apps should def declare #2018-07-30 05:34:27 alexmiller Generally you don’t want your libs deciding the Clojure version of the context they’re used in #2018-07-30 05:34:40 seancorfield Yeah, I was kinda coming at this from the app p.o.v.: it chooses the Clojure version and the libs just deal with it 🙂 #2018-07-30 05:35:00 alexmiller Which is effectively what happens #2018-07-30 05:35:40 alexmiller As top deps have priority #2018-07-30 05:36:03 alexmiller So in app context, their choice is most important #2018-07-30 05:36:41 seancorfield Pretty much all the libs out there should use "provided" with Clojure and apps then drive it. But that doesn't seem to get much traction 🙂 #2018-07-30 05:37:33 alexmiller Yeah, I don’t think that’s a battle worth fighting #2018-07-30 05:58:57 cfleming But t.d.a is an app, right? #2018-07-30 06:01:52 seancorfield I've only ever used it as a library... #2018-07-30 06:06:00 seancorfield The clj script invokes one of the -main functions in t.d.a. tho' (resolve tags, generate manifest, make classpath) but I'm not sure I'd consider tho' an app tho'? #2018-07-30 06:42:57 alexmiller It’s both in this case I think #2018-07-30 06:44:23 alexmiller But brew-install builds an uberjar so it’s really the app-maker #2018-07-30 16:39:33 kenny How are you guys debugging dependency conflicts? #2018-07-30 16:40:33 ghadi clojure -Stree for a start @kenny #2018-07-30 16:41:03 kenny Doesn't show the decision made in selecting the dependency. #2018-07-30 16:45:57 ghadi from what I've seen: the chosen dependency should only appear once in that tree, and its only parent is the driver of version selection #2018-07-30 16:47:14 kenny Correct. But I'd like to know the other parent dependencies that also have the conflicting child dependency. #2018-07-30 19:36:34 alexmiller There is a hidden verbose mode that sheds some additional light #2018-07-30 19:38:31 alexmiller clj -Sdeps ‘{:aliases {:v {:verbose true}}}' -A:v #2018-07-30 19:39:47 alexmiller Likely to be confusing to some degree as it’s mostly designed to let me debug stuff right now #2018-07-30 19:40:36 alexmiller I make zero promises about it’s existence or operation :) #2018-07-30 19:44:29 j0ni hehe do I detect a tiny hint of revelation regret 😄 #2018-07-30 20:13:05 alexmiller Just making it clear that is a not even alpha feature :) #2018-07-30 22:56:06 gfredericks I think for that we have to drop down to the hebrew alphabet and call it an aleph feature #2018-07-30 23:17:52 cfleming If we require people to do clj -א:v I’m pretty sure not many will find it. #2018-07-30 23:19:29 gfredericks perfect #2018-07-30 23:43:09 cfleming Several Cursive users have requested that Cursive show alongside the available aliases whether they’re system, user or project aliases. I can’t do that at the moment because there’s no way to tell which config file is which (short of something awful like guessing based on which is under the user’s home or something). #2018-07-30 23:44:13 cfleming One thing that would fix that would be if -Sdescribe returned:
:config-files { :system "/usr/local/Cellar/clojure/1.9.0.381/deps.edn" :user "/Users/colin/.clojure/deps.edn" }
#2018-07-30 23:44:28 cfleming Any love for that idea? #2018-07-30 23:49:29 seancorfield Changing -Sdescribe at this point will break a bunch of other tooling that relies on the format of the (clojure-env) call I think? #2018-07-30 23:49:55 gfredericks still alpha! #2018-07-30 23:50:02 cfleming No doubt. #2018-07-30 23:50:20 seancorfield Also, there's no reason that t.d.a. is limited to those three specific EDN files -- you can merge any number of EDN files to produce your environment... #2018-07-30 23:50:27 cfleming I guess following Rich’s latest talk that should be :config-files2 #2018-07-30 23:50:32 seancorfield 🙂 #2018-07-30 23:50:40 cfleming You could put any number of keys in that map too. #2018-07-30 23:51:06 gfredericks -Sdescribe2 #2018-07-30 23:51:13 cfleming I love it. #2018-07-30 23:51:16 ghadi it's not a breaking change at all #2018-07-30 23:51:47 ghadi you cannot change :config-files but you could enrich the map #2018-07-30 23:52:49 seancorfield :config-file-map 🙂 #2018-07-30 23:53:26 ghadi I ran into only being able to do -Sdeps once per invocation today #2018-07-30 23:53:37 cfleming Hehe, or :config-file-type indexed the other way around, so the file path goes to a keyword describing its type. #2018-07-30 23:53:39 ghadi this is related to the "how many config files are being merged?" question #2018-07-31 06:21:42 dominicm @cfleming I've opened a ticket for this. It was a map at some point. #2018-07-31 06:50:30 alexmiller Currently the config files are treated generically by the code as a list of dep sources. We now have several places that need to know the semantic meaning of different config files so they can be included or not, located, etc. (-Srepro, local dep path resolution, env, etc). This also interacts with whether the install level config is an explicit file or embedded as a resource inside tda. Something definitely needs to change but I need a clear space to sit down and think through all the details, so it’s coming. #2018-08-02 16:09:58 martinklepsch Are there any restrictions to where tools.deps will look for the :sha provided? #2018-08-02 16:10:24 ghadi what do you mean? #2018-08-02 16:11:37 alexmiller where else would it look other than the git repo? #2018-08-02 16:13:56 martinklepsch clj -Sdeps '{:deps {org.clojure-grimoire/lib-grimoire {:git/url "" :sha "767d68f605e5ec2ab740ae42cfc7b90a2948713c"}}}'
This fails with Manifest type not detected when finding deps for org.clojure-grimoire/lib-grimoire in coordinate {:git/url "", :sha "767d68f605e5ec2ab740ae42cfc7b90a2948713c"}
In a more involved setup it fails with the following error/stacktrace : #2018-08-02 16:14:42 martinklepsch #2018-08-02 16:16:35 ghadi the stack trace is a bit revealing... did you make edits to the file in .gitlibs/libs/... ? #2018-08-02 16:17:02 ghadi I cloned and found that commit #2018-08-02 16:17:24 alexmiller oh yeah, weird - github didn’t find it #2018-08-02 16:18:18 ghadi (We should consider making the checkout dirs readonly) #2018-08-02 16:18:36 dominicm @ghadi how do you make dirs read only? 😄 #2018-08-02 16:18:45 dominicm I guess you can chmod on linux, to prevent self-harm #2018-08-02 16:19:09 alexmiller I got your second error trying to run your first command #2018-08-02 16:20:33 martinklepsch haha #2018-08-02 16:21:00 martinklepsch I didn’t do any manual edits in .gitlibs (as far as I’m aware at least 😛) #2018-08-02 16:21:54 ghadi I found it super easy to accidentally jump to source in Emacs to a gitlibs SHA dir. #2018-08-02 16:22:11 ghadi make edits, hit save, break Content-Addressing guarantees #2018-08-02 16:23:05 dominicm I've done the same thing to real jars #2018-08-02 16:23:17 dominicm I had to uninstall zip so vim wouldn't let me :w inside a jar. #2018-08-02 16:23:24 dominicm I should fix it properly #2018-08-02 16:22:14 alexmiller I think the second error is the real error (the MissingObjectEx) #2018-08-02 16:22:32 alexmiller and then after that, it looks like you have a checkout directory, but it didn’t finish so it’s missing files (like the manifest files) #2018-08-02 16:22:35 martinklepsch I don’t even know how to jump to source so guess I’m safe 😄 #2018-08-02 16:22:47 martinklepsch should I try nuking .gitlibs? #2018-08-02 16:22:58 alexmiller I think you’ll see the same problem again #2018-08-02 16:23:02 ghadi no, not if Alex repro'ed immediately. #2018-08-02 16:23:10 alexmiller ^^ #2018-08-02 16:23:22 martinklepsch right, fair enough 🙂 #2018-08-02 16:24:26 bronsa i can confirm reproing w/o any particular setup #2018-08-02 16:24:47 martinklepsch Interestingly there are some files in that SHA’s gitlib dir (LICENSE and README) #2018-08-02 16:25:21 alexmiller yeah, it got part way done #2018-08-02 16:25:37 ghadi ➜ lib-grimoire git:(develop) git cat-file -t c7cd4f50871b20a580a66e77a34bfa9d336fe430
fatal: git cat-file: could not get object info
#2018-08-02 16:25:39 ghadi repush #2018-08-02 16:25:43 alexmiller I guess a) there is either something weird in this repo or a bug in jgit #2018-08-02 16:26:05 alexmiller and b) in this kind of a failure, we should nuke the checkout dir #2018-08-02 16:28:01 alexmiller a) is worth a bit more investigation #2018-08-02 16:28:12 alexmiller maybe there’s a force push or something weird there? #2018-08-02 16:28:31 ghadi There is something weird in this repo. #2018-08-02 16:28:33 martinklepsch Would love a tarball deps procurer btw 😛 #2018-08-02 16:28:35 ghadi ➜ lib-grimoire git:(develop) git cat-file -p HEAD^{tree}
100644 blob 3bc48030cba1b0f250f8b3d2cb0a2635f5195339 .gitmodules
100644 blob 7689f30efd6dcaadfb4033a444fa1c3f1848dba3 LICENSE
100644 blob c87d64a78fcaebbdfe02f188168a3270962869a1 README.md
040000 tree 4c51b4bd0c9980a782b29286cdeefdb49e7bd899 bin #### THIS
100644 blob 54f9aeba51d4b4a61c9ad3de046e770859a217c1 deps.edn
100644 blob f60a2b75223a9c8739d525ea62ba48aeeab878b0 project.clj
040000 tree 97a002c7db97cd89dc74606b77de343d2dbc878d resources
040000 tree 28ffc92968bebaf4930b08ff59d9ff5a3a5c297b src
040000 tree f4581b6f130115e5f0035945fc4ef6036076ba88 test
➜ lib-grimoire git:(develop) git cat-file -p 4c51b4bd0c9980a782b29286cdeefdb49e7bd899 ## Drill in
160000 commit c7cd4f50871b20a580a66e77a34bfa9d336fe430 hooks
100644 blob f3acf21ed723011f2586416e842ce41683bf3f29 install-hooks.sh
#2018-08-02 16:29:19 ghadi The second command cats out the SHA for the tree representing the bin directory, and it's pointing to a commit blob mysteriously #2018-08-02 16:29:36 ghadi I expect only blobs and trees to be children of trees #2018-08-02 16:30:02 martinklepsch I did force push to that branch, and it might have even been after I tried to use another revision from that repo — but seems weird that this would still replicate on a system that hasn’t had a clone yet? #2018-08-02 16:32:42 alexmiller I agree with Ghadi that that looks like a bad git structure #2018-08-02 16:33:12 ghadi try to overwrite bin/hooks and recommit it #2018-08-02 16:36:03 martinklepsch Deleted bin/ entirely & commited as 9810f7b20c1a676060f939cfd8f2f785f4dea3c7 but still same issue #2018-08-02 16:36:12 ghadi fetching.... #2018-08-02 16:36:33 martinklepsch did you have something else in mind when you wrote “overwrite”? #2018-08-02 16:37:19 ghadi repo looks good to me, you updated to 9810f7b20c1a676060f939cfd8f2f785f4dea3c7? #2018-08-02 16:37:59 martinklepsch oh no wait #2018-08-02 16:38:33 martinklepsch seems to work now! #2018-08-02 16:38:43 ghadi cool #2018-08-02 16:38:55 ghadi I don't know how that commit object got into the repo! #2018-08-02 16:39:03 martinklepsch just saw an exception and was assuming it’s the same but it was a different one 🙂 #2018-08-02 16:39:05 ghadi git fsck didn't complain at all... #2018-08-02 16:39:19 ghadi but that might not be looking for this prob #2018-08-02 16:40:37 alexmiller I was just doing that myself :) #2018-08-02 16:41:27 martinklepsch gotta say, I’m impressed by your git-fu 😄 #2018-08-02 16:41:49 alexmiller you’ve impressively horked this repo, kudos :) #2018-08-02 16:42:08 martinklepsch don’t look at me, I’m pretty sure it was @arrdem 🙂 #2018-08-02 16:42:35 alexmiller fair enough :) #2018-08-02 16:42:52 alexmiller just change my “you” ref via force-push #2018-08-02 16:44:41 martinklepsch an aside to this: are there any plans to add more procurers? A tarball procurer would have been cool in this situation 🙂 #2018-08-02 16:45:36 dominicm I've been hoping that a procurer extension mechanism would appear, off the back of the work to locate the owner of a namespace. #2018-08-02 16:45:54 alexmiller seems pretty far down the priority list #2018-08-02 16:46:17 alexmiller why not just the existing jar support? #2018-08-02 16:46:35 dominicm I imagine because github provides a tar.gz/zip on the releases page #2018-08-02 16:46:45 martinklepsch 👆 this #2018-08-02 16:47:04 alexmiller well, feel free to file an enhancement, preferably with a use case #2018-08-02 16:51:28 ghadi seems like a weird use-case #2018-08-02 16:53:17 alexmiller I can envision how to get a repo into this state via the plumbing api but I find it harder to envision how it happened via the porcelain #2018-08-02 16:53:46 alexmiller I assume some kind of bad rebase #2018-08-02 16:53:53 alexmiller but geez #2018-08-02 16:55:16 alexmiller and ghadi, it does seem like Git handles this better than jgit so maybe you’d never even see it shelling out #2018-08-02 17:35:22 lilactown I’m trying to patch juxt/pack to allow me to pass in an alias to use while generating a jar #2018-08-02 17:36:34 lilactown this is the relevant bit of code that is catching fire atm:
(let [deps-map (tools.deps.reader/slurp-deps (io/file deps))
resolve-args (tools.deps/combine-aliases
deps-map
(get-in parsed-opts [:options :alias]))]
(tools.deps/resolve-deps deps-map resolve-args)
#2018-08-02 17:37:10 lilactown I’m currently getting: org.apache.maven.model.resolution.UnresolvableModelException: Could not find artifact org.apache:apache:pom:1
3
#2018-08-02 17:37:31 dominicm @lilactown try adding :mvn/repos to your deps.edn #2018-08-02 17:37:59 dominicm See pack issues, just eating dinner else I'd give the full snippet #2018-08-02 17:39:49 lilactown that seemed to fix the error 🙂 thanks @dominicm #2018-08-02 17:41:00 dominicm @alexmiller to solve this, I'm considering inlining the system deps.edn into pack, do you think that would be a problem? #2018-08-02 17:41:29 lilactown (and it works!) #2018-08-02 17:49:05 alexmiller Not as long as you’re willing to roll with future changes #2018-08-02 18:20:27 dominicm Happy enough for now. #2018-08-02 18:41:39 seancorfield I originally had the system deps baked into boot-tools-deps then took it out in favor of calling (clojure-env) to get the list of deps.edn files to include -- which takes care of the :mvn/repos issue. #2018-08-02 18:41:55 seancorfield Does pack not use the system/user deps.edn files? #2018-08-02 18:42:39 dominicm @U04V70XH6 I don't want to do that in pack, I explicitly want to ignore the user deps.edn file, and I can't do that without (clojure-env) returning a map for the deps.edn files it considers. #2018-08-02 18:43:37 seancorfield You could at least assume the first entry in that vector was the system deps tho'... #2018-08-02 18:44:38 dominicm But that isn't part of the api, it could break at any point. I'd feel like a bad citizen depending on a coincidence rather than a contract. #2018-08-02 18:48:37 seancorfield I thought we were "guaranteed" that the vector of deps files was in order? Or do you mean that whole -Sdescribe thing isn't documented/guaranteed? /cc @alexmiller #2018-08-02 18:49:05 dominicm I recall asking and being told the order wasn't guaranteed. I asked when lein-tools-deps did this. #2018-08-02 18:53:20 seancorfield I must admit, I find that very surprising since order does matter ... #2018-08-02 18:54:56 dominicm @U04V70XH6 but being the first isn't attributed to system in anyway. A least-important override could be added at any time. #2018-08-02 20:17:58 alexmiller you can be guaranteed that they are in order: install, user, project, -Sdeps #2018-08-02 20:18:22 alexmiller however, the last 3 could all independently be either missing or present #2018-08-02 20:18:44 alexmiller and in future the first may possibly not exist #2018-08-02 20:31:30 seancorfield Thank you! #2018-08-02 21:06:44 dominicm If this guarantee is given, then I think the open ticket to convert it to a map is not needed. #2018-08-02 21:50:44 seancorfield Well, you still couldn't tell the difference between system + user and system + project (except, perhaps, by inspection of the actual paths?) and that was something @U0567Q30W was after I believe. #2018-08-02 21:51:51 seancorfield And you also wouldn't necessarily be able to tell the difference between system + user and user + project at the point where the system deps moved into the library itself (depending on how that is handled). #2018-08-02 21:52:47 alexmiller I think it’s useful to differentiate #2018-08-02 21:52:53 alexmiller for other needs too #2018-08-02 21:55:21 seancorfield Adding a new key to the result of -Sdescribe with some sort of map explaining where the deps files came from seems useful regardless of other concerns. I think if the system deps file moves into the codebase, then indicating that somehow in the list of config files, with a way to get the raw data structure, will also become important for tooling. #2018-08-03 02:06:44 cfleming The latest version of Cursive also bundles the system deps.edn BTW #2018-08-02 18:49:42 dominicm @alexmiller I might be being silly, but I don't see a way in the public api to slurp a dep which is a java.net.URL, e.g. io/resource. slurp-edn depends on being a file, and canonicalize-all-syms is a private function, so I cannot utilize it. #2018-08-02 20:24:08 alexmiller could probably break slurp-edn into something smaller that takes a reader (although the file-ness is being used to construct a good error message there) #2018-08-02 20:24:28 alexmiller canonicalize-all-syms is not public as it may go away #2018-08-02 20:25:12 alexmiller there’s now a canoncialization step built into the extension apis #2018-08-02 20:25:44 alexmiller you can of course invoke it via the private var if you want to call it for now #2018-08-03 21:10:18 emergence @alexmiller (or anyone else in the know) - I'm pulling down a company Clojure library from a private github repo via deps.edn -- had to tweak a few things to get auth to play nice, but now I have another issue: is there a way to indicate which jar file it will use? #2018-08-03 21:10:44 emergence I have one that will include test dependencies and am looking to use that version instead #2018-08-03 21:20:46 ghadi what did you tweak for Auth @emergence? #2018-08-03 21:21:29 emergence I had to comment out my IdentityFile line in .ssh/config #2018-08-03 21:23:41 ghadi yeah this is a known issue #2018-08-03 21:23:57 ghadi we have a potential resolution for a host of auth related issues #2018-08-03 21:25:35 emergence I saw there is an open bug around this #2018-08-03 21:21:29 alexmiller deps.edn does not have any way to include test scoped transitive deps #2018-08-03 21:22:07 emergence Sorry I wrote incorrectly, I meant test namespaces not transitive deps #2018-08-03 21:22:37 alexmiller I was confused by you saying jar file above #2018-08-03 21:22:44 alexmiller in tandem with github repo #2018-08-03 21:23:02 alexmiller or rather, I am still confused. perhaps you can back up. :) #2018-08-03 21:24:30 emergence so in my deps.edn I have something like
mycompany/depname {:git/url " #2018-08-03 21:24:56 emergence And forgive me if I'm overlooking something silly I'm trying tools.deps.alpha for first time s #2018-08-03 21:25:59 emergence I want to access test namespaces from depname #2018-08-03 21:26:43 alexmiller in that project’s deps.edn, you could include :paths ["src" "test"] #2018-08-03 21:26:49 alexmiller but there is no way to do that externally #2018-08-03 21:27:06 alexmiller so I think my answer is still, you can’t do that #2018-08-03 21:29:11 emergence That's still helpful -- the project is still running off a project.clj file so I can see about merging in a secondary deps.edn for my purposes. #2018-08-03 21:31:42 alexmiller it is possible to use a :deps/root key to point to a subdir holding a deps.edn that included the test, but that’s likely to confuse people #2018-08-03 21:34:07 emergence If I did :deps/root "test" and stuffed the deps.edn in test/ do you think that would work? #2018-08-03 21:35:39 emergence and then specified the appropriate relative paths #2018-08-03 21:41:52 emergence Either way I'll play around with it and see what works for my team -- thanks for spending a few mins chatting on this 🙂 #2018-08-04 20:07:03 lilactown how would I go about adding a dependency (Java) that uses gradle (via Github)? #2018-08-04 20:07:09 lilactown no pom to be found #2018-08-04 20:09:28 dominicm @lilactown generally it's deployed to a maven repository #2018-08-04 20:09:33 dominicm Even if it's using gradle #2018-08-04 20:10:29 lilactown ah. in this particular instance it’s not, since the code I want is actually a part of an application #2018-08-04 20:11:39 lilactown my hope is that I could bring it in as a dependency and play with it in clojure #2018-08-04 20:15:57 dominicm I don't think so in this case. t.d.a doesn't have a reader for gradle files. #2018-08-04 20:19:36 lilactown I guess I could fork it and generate a pom? #2018-08-04 20:21:15 dominicm or just install it locally via gradle #2018-08-05 10:53:07 michal hi. is there any way to force clj to run in offline mode (to not fetch newest snapshots in particular) ? #2018-08-05 20:28:11 alexmiller Currently, no, but planning to add #2018-08-05 15:27:50 lilactown is add-lib going to make it’s way into master? #2018-08-05 20:27:56 alexmiller Yes #2018-08-06 23:19:36 kenny I have org.slf4j/slf4j-nop in my :exclusions for a dependency but it is still included in the tree printed out via clj -Stree. org.slf4j/slf4j-nop is a second level dependency (i.e. a dependency that my dependency depends on). Why is the org.slf4j/slf4j-nop not getting excluded? #2018-08-06 23:36:53 hiredman what does -Spath say? #2018-08-06 23:50:31 kenny It also includes org.slf4j/slf4j-nop. #2018-08-07 00:11:42 kenny Deleted .cpcache and it fixed it. #2018-08-07 07:24:56 dottedmag Sounds like a bug. Why it was not invalidated on its own? #2018-08-07 11:50:26 alexmiller The only cache staleness bug I’m aware of is with local deps #2018-08-07 11:51:13 alexmiller Where downstream deps.edn that change don’t force a cache invalidation #2018-08-07 17:32:13 martinklepsch I would really like to fix https://dev.clojure.org/jira/browse/CLJ-2037 but if I understand correctly it’s tricky with the current registry structure since keywords can’t carry additional metadata. Would be happy about any suggestions. Maybe @alexmiller can also comment to what degree this feature warrants a restructuring of the registry. #2018-08-07 17:34:19 martinklepsch (I assume the patch Felix Andrews describes in his comment won’t work with specs like (s/def ::thing ::other-thing) but I haven’t fully checked IIRC) #2018-08-07 17:37:59 alexmiller definitely on board with fixing it, but not going to happen until Rich has a chance to get back to spec work, and that will likely be a while #2018-08-07 17:41:58 martinklepsch Got it. Guess I’ll run with a fork that has a fix for this issue then. #2018-08-07 18:33:56 dnolen is it not possible to pass JVM opts via :main-opts? #2018-08-07 18:35:30 dominicm That's what jvm-opts is for? #2018-08-07 18:39:30 dnolen ah thanks, I keep looking for stuff in the guide instead of the reference #2018-08-07 18:41:46 dnolen aliases don’t compose across deps.edn files right? #2018-08-07 18:47:18 seancorfield No, later aliases replace earlier ones with the same name. #2018-08-07 18:47:56 seancorfield (I think that's the safer behavior, TBH) #2018-08-07 18:48:24 dnolen yeah not saying it’s desirable #2018-08-07 18:49:08 seancorfield Heh, I've wished for composable aliases from time to time... but always stop myself from creating a JIRA ticket 🙂 #2018-08-08 12:20:46 rickmoynihan is it (or will it ever be) possible to put specific paths from inside a git dep on the class/resource path, without needing a deps.edn file at the project root? #2018-08-08 12:20:54 rickmoynihan thinking for resource deps #2018-08-08 12:26:48 alexmiller No? #2018-08-08 12:37:09 dominicm @rickmoynihan you could probably use gitlibs directly! #2018-08-08 12:42:35 rickmoynihan @dominicm: how do you mean? #2018-08-08 12:43:06 dominicm I suppose not if you wanted them on the classpath 🤔 #2018-08-08 12:45:02 rickmoynihan Thanks hadn’t seen gitlibs before #2018-08-08 12:46:12 rickmoynihan it’s ok I think :local/root is better for this anyway #2018-08-08 13:01:03 alexmiller tools.deps uses gitlibs for its git deps #2018-08-08 18:58:54 richiardiandrea has anybody run into problems with OpenJDK 9 and retrieving deps? #2018-08-08 19:15:40 lilactown is there a way to tell the CLI tools to “just download dependencies”? #2018-08-08 19:15:43 lilactown trying to tune my docker build #2018-08-08 19:17:25 dominicm As opposed to doing what #2018-08-08 20:19:46 richiardiandrea reporting the full error here, is it a known issue or it is worth a Jira? #2018-08-08 20:56:10 alexmiller that’s different #2018-08-08 20:56:32 alexmiller the problem here is buried - “Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact mount:mount:pom:0.1.13 from/to clojars (https://repo.clojars.org/): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target” #2018-08-08 20:56:49 alexmiller this is a certificate problem re the JDK being used and clojars #2018-08-08 21:01:12 alexmiller openjdk9 does not install with any root certs (I think this is fixed in 10) #2018-08-08 21:49:48 richiardiandrea oh cool ok, thanks a lot Alex #2018-08-08 20:24:53 ghadi @lilactown you can put RUN clojure -Spath in your dockerfile #2018-08-08 20:25:11 ghadi should have the intended effect, if an oblique approach #2018-08-08 21:39:17 lilactown @ghadi thanks! I think that almost worked… for some reason my uberjar command still downloads and installs all it’s deps 😭 #2018-08-08 21:39:50 ghadi lein uberjar? #2018-08-08 21:40:14 lilactown not quite:
:uberjar
{:extra-deps
{pack/pack.alpha
;; {:local/root "/Users/will/Code/pack.alpha"}
{:git/url ""
:sha "c444ae3af083b0bfa68160daac576607232f556c"}}
:main-opts ["-m" "mach.pack.alpha.capsule" "dist/lilactown.jar"
"-a" ":server" "-m" "lilactown.core"]}
#2018-08-08 21:40:47 ghadi All its deps, or just the pack deps? #2018-08-08 21:41:17 lilactown I think it’s just the pack deps #2018-08-08 21:41:25 ghadi RUN clojure -A:uberjar -Spath #2018-08-08 21:41:39 ghadi should do the the trick to handle that #2018-08-08 21:42:00 lilactown here’s my relevant dockerfile stuff:
COPY deps.edn .
RUN clojure -A:server:uberjar -Spath
COPY . /usr/app
RUN clojure -A:uberjar
#2018-08-08 21:43:12 lilactown hrm. I wonder if changing it to -R:server:uberjar -Spath would change it #2018-08-08 21:43:15 lilactown seems more correct anyway #2018-08-08 21:43:17 ghadi are you setting WORKDIR /usr/app in the dockerfile? #2018-08-08 21:43:26 ghadi -A subsumes -R #2018-08-08 21:44:03 lilactown yes, I’m setting workdir too #2018-08-08 21:47:11 ghadi (I like multistage builds)
I'm surprised L27 redownloads things, to be honest. All should get downloaded on L21 #2018-08-08 21:48:45 lilactown yeah me too 😕 #2018-08-08 21:49:12 ghadi offtopic for this room, but I wouldn't trust openjdk-8-alpine until OpenJDK fully supports Alpine #2018-08-08 21:49:40 ghadi JDK 11 has official support, IIRC #2018-08-08 21:49:45 lilactown 🤔 I haven’t noticed any problems yet. anything in particular I should watch out for? #2018-08-08 21:50:41 ghadi not sure tbh. I've also used it in the past. Now I use jlink --add-modules to create slimmer packaged JVM #2018-08-08 21:52:36 ghadi FROM openjdk:10 as jdk
RUN jlink \
--strip-debug \
--compress=1 \
--vm=server \
--output /tmp/linkedjdk \
--add-modules ,jdk.management,jdk.unsupported,java.xml.bind,java.xml.ws.annotation
FROM debian:stretch-slim
COPY --from=jdk /tmp/linkedjdk /jdk/
# This is temporarily necessary as it appears the new OpenJDK TrustStore is incomplete
# Apparently, AWS's Step Function endpoint uses a different signing root than S3
COPY --from=jdk /docker-java-home/lib/security/cacerts /jdk/lib/security/
COPY my.jar /MYCOMPANY/
ENTRYPOINT ["/jdk/bin/java"]
#2018-08-08 21:52:45 ghadi that's my dockerfile @lilactown #2018-08-08 21:53:11 lilactown slick #2018-08-08 21:53:28 lilactown saving this for later 😉 I remember you talking about this last week I think #2018-08-08 21:55:46 ghadi depending on what modules you need, you can get by with less #2018-08-08 21:51:21 ghadi I'd assume it would manifest as a hard VM crash. But if it works, it works! #2018-08-09 17:44:08 richiardiandrea uhm, this baffles me, what am I doing wrong?
clojure -Srepro -Sdeps '{cljs-tooling {:mvn/version "0.3.0-SNAPSHOT" :exclusions [org.clojure/clojure org.clojure/clojurescript]}}' -Spath
src:/home/arichiardi/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar:/home/arichiardi/.m2/repository/org/clojure/spec.alpha/0.1.143/spec.alpha-0.1.143.jar:/home/arichiardi/.m2/repository/org/clojure/core.specs.alpha/0.1.24/core.specs.alpha-0.1.24.jar
#2018-08-09 17:45:05 richiardiandrea in particular I am not seeing the dep I want to include #2018-08-09 18:28:19 dominicm @richiardiandrea is it the same for other dependencies, or just that one? I'm wondering about -Srepro and -Sdeps #2018-08-09 18:30:36 richiardiandrea uhm dunno let me try smth else #2018-08-09 18:31:20 ghadi Yeah I'm not sure how those interact together... #2018-08-09 18:31:34 richiardiandrea $ clojure -Srepro -Sdeps '{mount {:mvn/version "0.1.13"}}' -Stree
org.clojure/clojure 1.9.0
org.clojure/spec.alpha 0.1.143
org.clojure/core.specs.alpha 0.1.24
#2018-08-09 18:31:51 ghadi -Srepro might negate Sdeps #2018-08-09 18:31:53 richiardiandrea oh, maybe I need :deps #2018-08-09 18:32:16 richiardiandrea riiiiight #2018-08-09 18:32:25 richiardiandrea $ clojure -Srepro -Sdeps '{:deps {mount {:mvn/version "0.1.13"}}}' -Stree
org.clojure/clojure 1.9.0
org.clojure/spec.alpha 0.1.143
org.clojure/core.specs.alpha 0.1.24
mount/mount 0.1.13
#2018-08-09 18:33:08 ghadi Oh, yeah that would help #2018-08-09 18:34:07 richiardiandrea lol #2018-08-10 04:38:10 steveb8n is there a lib that provides a watcher (e.g. re-runs tests or a specific fn when it sees a file change) that works with deps.edn? I can’t find anything #2018-08-10 05:02:06 seancorfield @steveb8n Wouldn't that just be any command-line watcher than can (re-)run another command when files change? #2018-08-10 05:02:35 steveb8n probably but I’ve only used the test watcher #2018-08-10 05:02:41 steveb8n watchers #2018-08-10 05:02:46 seancorfield The clj script is a one-shot runner that loads deps.edn and runs whatever :main-opts are specified. #2018-08-10 05:03:17 steveb8n and the test watchers are all built for lein/boot #2018-08-10 05:03:19 seancorfield The lein/`boot` test watchers are built to re-run tests -- so it's very tightly coupled. #2018-08-10 05:04:31 steveb8n I could build my own, load it into classpath from a gist but I’d rather avoid the work if it’s already been done #2018-08-10 05:05:34 seancorfield I actually think test watchers are a bit of an anti-pattern... #2018-08-10 05:06:15 seancorfield They sort of imply an external process -- external to your editor and your REPL -- that you have to keep watching while you are editing. #2018-08-10 05:10:40 seancorfield When I'm working on code, I'm running code in the REPL, and I can run any tests in my editor with a hot key.... I wouldn't expect to re-run my entire test suite on every minor code change and file save. #2018-08-10 05:11:05 seancorfield And my workflow usually involves evaluating code without even saving files anyway ¯\(ツ)/¯ #2018-08-10 05:12:04 seancorfield I can evaluate forms to the REPL without saving the file so why would I bother with that step? 🙂 #2018-08-10 05:13:02 steveb8n I agree although I’m generating a diagram using dorothy and I have to switch from the layout ns to the driving ns every time I want to see the change #2018-08-10 05:13:23 steveb8n would be much faster during dev to see updates without the switch #2018-08-10 05:13:33 dominicm Krei is already watching the classpath for changes. No reason it couldn't run tests in a classloader or something. #2018-08-10 05:13:47 steveb8n cool. I’ll take a look #2018-08-10 05:14:01 dominicm It doesn't do that right now though. #2018-08-10 05:14:31 steveb8n doh, bait-in-switch 🙂 #2018-08-10 05:16:13 steveb8n in that case I’ll carry on switching ns for now. maybe one of the test runners will get this soon. #2018-08-10 05:16:33 steveb8n thanks @seancorfield @dominicm for the thoughts #2018-08-10 05:20:34 seancorfield @steveb8n One option to consider is to have a (comment ...) form in your layout ns that calls functions in your driving ns, then you can do everything in one namespace. #2018-08-10 05:22:10 seancorfield I tend to have a (comment ...) block at the end of a lot of my ns's that let me run code to setup/test/teardown the code in that ns. #2018-08-10 05:27:57 steveb8n I have that but most of my iterative changes are in the driven ns, not the driving ns. I’ll try that idea in the driven ns though - might work #2018-08-10 05:29:52 steveb8n that works! good idea, thanks. now re-loading the driven ns invokes the fn in the driver ns #2018-08-10 05:30:05 steveb8n the simplest ideas are always best #2018-08-10 06:11:55 seancorfield @steveb8n Cool. I've gotten a lot more productive by dropping (comment ...) blocks all over the code base so I can stay in one file and run whatever code I need (and because it isn't compiled at load time, you can have circular references in such blocks). I use it for code that starts/stops Components, runs tests, sets up state etc. #2018-08-10 06:12:36 steveb8n yeah, it was the circular ref that was blocking my brain from trying that. great trick #2018-08-10 06:12:49 steveb8n my fingers thank you #2018-08-10 10:16:37 thheller @alexmiller I'm looking into using the tools.deps.alpha CLJ API as the primary dependency resolver for shadow-cljs (was using pomegranate before). resolve-deps works nicely but I noticed that make-classpath just concats all paths together without sorting them properly. does the CLI itself also use this behavior? #2018-08-10 10:17:07 thheller order shouldn't matter but in case there are any conflicts on the classpath I'd rather have something deterministic rather than relying on the hash ordering of the lib-map #2018-08-10 10:17:59 dominicm @thheller This is intentional, my understanding is that the statement is "order doesn't matter". #2018-08-10 10:18:44 thheller yeah just might be me running into issues with pomegranate where order definitely mattered #2018-08-10 10:18:52 thheller but it didn't resolve version conflicts properly, so shouldn't happen in tdeps #2018-08-10 12:31:51 alexmiller Order should only matter if you have two dependencies that contain the same thing #2018-08-10 12:32:08 alexmiller And if you have that something is wrong #2018-08-10 12:32:46 alexmiller Maven uses order for selection but we use version #2018-08-10 13:48:26 mhuebert is there any way to have multiple git dependencies in the same git repo, which depend on each other? eg. say I publish https://github.com/acme/MonsterLib, which contains two projects, A and B, in subdirectories, so we have /A/deps.edn and /B/deps.edn, and B depends on A. For local dev, I can put a :local/root entry in /B/deps.edn which points to "../A", but (as far as I know) this won’t be properly required by a consumer of B #2018-08-10 13:49:44 mhuebert maybe a clearer way of asking is, “is there a way for tools.deps to resolve relative :local/root (or equivalent) dependencies in a git repo” #2018-08-10 13:49:47 dominicm @mhuebert I think you're looking for :deps/root #2018-08-10 14:31:36 rickmoynihan So does :deps/root allow you to treat a sub project (i.e. a sub directory with a deps.edn file) within a git repo as a specific dep, instead of the top level project directory? Or am I misunderstanding? #2018-08-10 14:37:57 dominicm That's exactly it #2018-08-10 13:50:58 mhuebert looking for docs on that #2018-08-10 13:52:18 alexmiller I don’t think there are any docs but it’s a thing #2018-08-10 13:52:41 mhuebert would the entries be the same as with :local/root? ie. just put relative paths there #2018-08-10 13:52:47 alexmiller Yes #2018-08-10 13:53:02 mhuebert sounds like exactly what I was hoping for #2018-08-10 13:53:05 alexmiller Relative to local/root #2018-08-10 13:55:43 dominicm @alexmiller @mhuebert I think exactly as :local/root is wrong here, it would be just "A/" alone I think. #2018-08-10 14:15:36 mhuebert hmm, I must still have something wrong. I’ve made a simple example, a tree like:
.
├── App
│ └── deps.edn
└── Monster
├── A
│ └── deps.edn
└── deps.edn
(repo: https://github.com/mhuebert/deps-root-test)
App/deps.edn depends on {:local/root "../Monster"}, which depends on {:deps/root "A"}, but if I run clj -Spath in App I get Coordinate type not loaded for library monster/A in coordinate #:deps{:root "A"} #2018-08-10 14:16:42 dominicm @mhuebert you need to do a :git/url too, alongside the :deps/root #2018-08-10 14:18:49 mhuebert hmm. can Monster ever depend on the current version of A? #2018-08-10 14:18:54 mhuebert if they are in the same repo #2018-08-10 14:20:15 mhuebert (if you need the sha, which you don’t have until you have made the commit that would presumably contain that link) #2018-08-10 14:21:51 mhuebert yes, if :local/root was relative to its own location instead of the current working directory, then I could use that #2018-08-10 14:22:20 alexmiller it can’t be relative to itself #2018-08-10 14:22:45 dominicm @alexmiller relative to the manifest, no? #2018-08-10 14:23:07 alexmiller when you include a local dep by location, it can be relative to the project including the dep #2018-08-10 14:23:18 alexmiller instead of relative to cwd #2018-08-10 14:24:06 dominicm I think you're both saying the same thing, aren't you? Or am I misunderstanding? #2018-08-10 14:24:12 alexmiller maybe :) #2018-08-10 14:24:29 alexmiller I’m just trying to restate clearly for myself mainly #2018-08-10 14:25:35 mhuebert what I am wondering is, if that jira ticket was solved, could Monster/deps.edn contain a {:local/root "A"} pointer to the A subdirectory, and resolve for a consumer of Monster, eg. via :local/root (in the example above, App) or a git dep? #2018-08-10 14:27:11 alexmiller maybe? #2018-08-10 14:32:58 mhuebert I’m not sure how else one could consume multiple projects (which may have dependencies among themselves) from the same git repo #2018-08-10 14:34:15 mhuebert monorepo woes. #2018-08-10 14:41:46 mhuebert I’ve updated the example to be more simple/realistic,
├── App ;; user-land, wants to use project A
│ └── deps.edn
└── Monster ;; library-land, a repo or local dir
├── A
│ └── deps.edn
└── B
└── deps.edn
the motivating case being that you want to publish project A, which depends on project B, in the same Monster repo.
The test case is, can App depend on A via git or local/root, and have tools.deps successfully follow the dependency to B. #2018-08-10 14:51:31 alexmiller what’s not being tracked at all in the code is the directory basis and you want that to shift as you examine different local or git dirs #2018-08-10 14:52:00 alexmiller so that it’s relative to the currently focused lib #2018-08-10 14:52:37 alexmiller and then use it to resolve relative dirs #2018-08-10 15:00:07 alexmiller which was actually kind of the intent of :deps/root #2018-08-10 15:00:24 alexmiller local deps don’t actually use deps/root right now to detect the manifest (git deps do) #2018-08-10 19:59:43 dominicm Does it make sense for it to do so? Wouldn't you just provide a local path which included the deps/root? #2018-08-10 20:17:00 alexmiller yes, it makes sense, and no, I don’t think you would do that. You want to be able to specify a deps/root relative to the local root I think? but this is actually the less interesting part of the problem. #2018-08-11 05:36:30 dominicm I don't understand what additional functionality, beyond specifying a subdir, is provided by deps/root. What behaviour would be different? #2018-08-10 15:00:54 alexmiller so there are actually several issues here #2018-08-12 18:42:01 thheller @alexmiller does tools.deps support providing maven mirrors? in lein/pomegranate I could set :mirrors. Can't find anything related in tools.deps sources? #2018-08-12 18:42:47 alexmiller Not yet #2018-08-12 18:50:21 thheller m2/settings.xml is read either right? #2018-08-13 08:36:10 martinklepsch I haven't looked around much, but just in case this is a known thing, some folks building docs for cljdoc locally get an error like this
Error building classpath. /var/folders/gm/p9fh4d3n5m11nnyjrsry1cm80000gq/T/cljdoc-expound-expound-0.7.2-SNAPSHOT6141675865145266918/impl-src is not a relative path
Does that look familiar to anyone? Are :paths required to be relative? #2018-08-13 23:35:02 richiardiandrea sorry maybe an FAQ but can tools.deps read deps from a local folder? #2018-08-13 23:36:16 richiardiandrea oh :local/root awesome #2018-08-14 05:45:13 isaac can clj handle java files that in paths? #2018-08-14 05:47:07 dominicm No #2018-08-14 05:47:27 dominicm Or rather, not in source files. If they're from Maven it's okay. #2018-08-14 05:48:45 isaac can i call javac from clj? #2018-08-14 06:21:24 dominicm @isaac absolutely. #2018-08-14 06:21:27 dominicm I do this in pack. #2018-08-17 03:12:33 alex-dixon Going off isaac’s question from earlier, is there a good workflow for mixed java and clojure projects using deps.edn? #2018-08-17 10:33:00 jetzajac can I add a lib that consists of multiple jars on disk? #2018-08-17 12:07:33 alexmiller Generally, no. Typically a lib consists of a single artifact #2018-08-17 12:08:03 alexmiller There are ways to hack it by listing the jars explicitly in :paths though #2018-08-17 18:10:07 lilactown do multiple -Sdeps merge? #2018-08-17 18:30:46 lilactown answer: nope #2018-08-18 06:52:04 cfleming @alexmiller Ugh, is it expected that users will be able to add jars via :paths? I’ll have to add support for that if so. #2018-08-18 08:47:39 alexmiller I would not consider that to be normal practice #2018-08-19 15:30:20 sveri @alexmiller Hi, how about native support for windows, did you come to work on it? #2018-08-19 16:04:49 alexmiller I’ve done a port to power shell but have not had time to finish all the build and release stuff, which interacts with some of the script #2018-08-19 16:05:05 alexmiller Hoping to be able to refocus on that soon and finish it off #2018-08-19 18:44:11 sveri Ok, thanks for your work. I was trying to use cli on WSL, which works, but fighweel main for instance needs an x11 instance which I don't have installed right now.
#2018-08-20 08:01:04 levitanong hi all, is there a way to run clojure such that it only downloads the dependencies and exits? Similar to lein deps. Use-case is in docker, where caching is done by diffing USEd files. #2018-08-20 09:02:07 delaguardo clojure -e "(clojure-version)" #2018-08-20 09:02:44 delaguardo this will download everything, print out your clojure version and exit with exit code 0 #2018-08-20 09:07:32 levitanong @delaguardo haha wouldn’t clojure -e "(System/exit 0)" work equally well in that case? #2018-08-20 09:08:04 delaguardo sure) #2018-08-20 09:08:52 delaguardo I’m just copying my configuration where I’m using printed value as a part of deploy report) #2018-08-20 10:37:34 levitanong Okay, thanks @delaguardo! #2018-08-20 10:54:19 dottedmag clojure -e '' works too #2018-08-20 12:52:38 alexmiller clojure -Spath will work #2018-08-20 16:26:07 levitanong @dottedmag oh that’s nice and concise!
@alexmiller would clojure -Sdescribe also work? #2018-08-20 16:26:27 levitanong I ask because that would end up printing a lot less than -Spath #2018-08-20 16:26:35 ghadi clojure -Spath >/dev/null #2018-08-20 16:26:44 ghadi sufficient for a dockerfile ^ #2018-08-20 16:28:04 levitanong Haha @ghadi that’s an interesting solution, but I think I’d prefer clojure -e '' to that. #2018-08-20 16:28:39 ghadi that will launch the JVM twice, which is why alex suggested -Spath #2018-08-20 16:36:15 levitanong oooh, so like, once for getting the deps, and then another for launching the repl? #2018-08-20 16:36:40 ghadi si #2018-08-20 16:36:53 ghadi not a big deal #2018-08-20 16:42:00 levitanong still good to know how things work! Thanks for explaining 🙂 #2018-08-21 18:39:33 ghadi (Of course the absence of a convenience doesn't hold me back at all when I have a choice) #2018-08-21 18:55:31 alexmiller if you’re not using aliases that have M or O, then why do you care? #2018-08-21 19:02:56 ghadi a test alias is a good example of when you want to launch a repl with deps but without (let's say) main options for some test-runner #2018-08-21 19:09:37 ghadi tldr: because I am using an alias with M / O #2018-08-21 23:19:35 alexmiller I’m going to file this in the tiny violin category #2018-08-21 23:52:34 ghadi I'll take a tiny little 🎹 @alexmiller #2018-08-21 23:57:38 alexmiller Granted #2018-08-22 00:54:39 cfleming Now I have to create a Tiny Violin tag in my Github tracker. #2018-08-22 14:04:07 dominicm @alexmiller is this something you would want to support?
{:git/url "../"
:deps/root "sub-dir"
:sha "abcabc"}
Essentially mono-repos with version pinning. #2018-08-22 14:04:52 dominicm I'm getting:
Error building classpath. Destination path ".." already exists and is not an empty directory
org.eclipse.jgit.api.errors.JGitInternalException: Destination path ".." already exists and is not an empty directory
at org.eclipse.jgit.api.CloneCommand.verifyDirectories(CloneCommand.java:257)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:189)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:49)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
at clojure.tools.gitlibs.impl$ensure_git_dir$fn__850.invoke(impl.clj:109)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:105)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
at clojure.tools.deps.alpha.extensions.git$eval894$fn__896.invoke(git.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:238)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:168)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:152)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:215)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invokeStatic(make_classpath.clj:59)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invoke(make_classpath.clj:52)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:70)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
which I suspect is due to how the folder name is being chosen. #2018-08-22 14:07:43 dominicm oh, weird, I can't supply an absolute path either, it is forced to be a relative one. #2018-08-22 14:08:06 dominicm Looks like paths aren't supported at all: Error building classpath. org.eclipse.jgit.transport.TransportLocal cannot be cast to org.eclipse.jgit.transport.SshTransport #2018-08-22 14:08:09 dominicm I'll open JIRA tickets #2018-08-22 14:21:54 alexmiller local git repos were intentionally out of scope #2018-08-22 14:22:45 alexmiller could possibly be added, but I would rank that below a lot of other things to work on #2018-08-22 14:22:59 alexmiller so I’ve classified these as minor enhancements in jira #2018-08-22 14:29:01 dominicm sounds very reasonable to me. I just don't touch the priority fields, based on what I think I'm supposed to do. #2018-08-22 16:07:05 onetom @dominicm the whole point of a monorepo is that you don't have to worry about the versions, because you are supposed to ensure that on every commit all modules/libraries/apps still build and work in your monorepo 😕 #2018-08-22 16:07:49 dominicm @onetom I know 😞 sometimes we put things into life support though, quite useful under those conditions. #2018-08-22 16:10:44 onetom I was wondering how can I control the refresh period of -SNAPSHOT dependencies.
Or is it possible to pin them to a specific version?
For example, I have all these versions:
$ ls -lah ~/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/*.jar
-rw-r--r-- 1 onetom staff 135K Aug 14 20:07 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-20180813.135516-2.jar
-rw-r--r-- 1 onetom staff 137K Aug 15 19:16 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-20180814.194035-3.jar
-rw-r--r-- 1 onetom staff 140K Aug 21 08:29 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-20180818.154912-4.jar
-rw-r--r-- 1 onetom staff 140K Aug 21 08:29 /Users/onetom/.m2/repository/com/bhauman/figwheel-main/0.1.8-SNAPSHOT/figwheel-main-0.1.8-SNAPSHOT.jar
I would be happy to just stick to figwheel-main-0.1.8-20180818.154912-4.jar until im on a slow internet connection and extra network traffic can cause several seconds of extra delay #2018-08-22 16:12:02 dominicm I think that "0.1.8-20180818.154912-4" works as a version #2018-08-22 16:14:50 onetom @dominicm thanks, indeed it does.
so i can just specify explicit versions of the transitive dependencies too and it should be fine. #2018-08-22 16:15:51 onetom still, it would be good to be able to disable automatic SNAPSHOT updating somehow. iirc i could do that in boot #2018-08-22 16:19:22 dominicm in boot I pass --offline #2018-08-22 16:21:17 onetom yeah, that's the option i was thinking about! #2018-08-26 18:21:30 bhauman Is there a central list of tools.deps tools like packr? #2018-08-26 18:34:12 dominicm J/w Do you mean pack.alpha? #2018-08-26 18:22:38 bhauman I remember there was one called mayven or something like that and I can’t seem to find it out there on webs #2018-08-26 18:22:49 seancorfield There's a partial list on repo's wiki #2018-08-26 18:24:08 bhauman Do you remember the mayven project? I though metosin did it but nope. #2018-08-26 18:24:18 bhauman It’s not on the wiki #2018-08-26 18:24:43 bhauman thanks for pointing out that page though #2018-08-26 18:26:08 seancorfield I don't recall mayven (but I'm terrible with names) #2018-08-26 18:26:46 bhauman good pin #2018-08-26 18:26:59 bhauman and its a horrible name to search for #2018-08-26 18:27:48 seancorfield Yeah, Bing isn't turning up anything for me with mayven in it. #2018-08-26 18:45:20 alexmiller I recall it but don’t have a link. I don’t remember it being a tools.deps thing? #2018-08-26 19:11:54 bhauman I thought it was a tools deps thing but that may be why I’m not finding it #2018-08-26 19:53:18 dominicm @bhauman ^ #2018-08-26 21:37:41 bhauman @dominicm Thanks! #2018-08-27 01:17:52 steveb8n has anyone else used cognitect.test-runner in CI? I’m finding it returns status zero when there are test failures #2018-08-27 01:26:03 seancorfield @steveb8n Are you sure? I use it with clojure.java.jdbc and I'm pretty sure I get a non-zero status when there are failures. #2018-08-27 01:26:10 seancorfield Are you using clojure or clj? #2018-08-27 01:26:18 steveb8n I’m using clojure #2018-08-27 01:26:35 steveb8n is that a possible explanation? #2018-08-27 01:27:04 steveb8n I can reproduce on my laptop also. have looked at source and it doesn’t jive #2018-08-27 01:27:16 seancorfield clojure is the underlying script so it should be safe(r) anyway. #2018-08-27 01:27:27 seancorfield Are you on an older version of clj/`clojure`? #2018-08-27 01:28:49 steveb8n nope. 1.9.0 in deps.edn #2018-08-27 01:29:08 steveb8n I might try debug in Cursive and see if that explains it #2018-08-27 01:29:11 steveb8n weird #2018-08-27 01:31:15 cfleming @steveb8n I think @seancorfield is talking about the clj/`clojure` CLI scripts, not Clojure itself. #2018-08-27 01:31:39 steveb8n oh good point. I’ll check #2018-08-27 01:32:29 steveb8n 1.9.0.375 #2018-08-27 01:35:00 steveb8n when I run from repl I see correct results so I think Sean is right in that this is maybe the scripts masking the system/exit value #2018-08-27 01:41:12 steveb8n just updated clojure using brew. same result sadly. will keep digging #2018-08-27 01:43:20 steveb8n just spotted this commit. I’m on an old version of the test runner. testing new sha now #2018-08-27 01:44:23 steveb8n that was it. thanks for the sounding board 🙂 #2018-08-27 01:47:08 steveb8n interesting challenge to keep the SHA’s in readmes current when a jar/snapshot not available in clojars. not sure how that can be improved. #2018-08-27 02:25:01 alexmiller rub some script on it #2018-08-27 02:30:19 seancorfield @steveb8n Sorry, stepped away. Glad you got it figured out. Not sure why a SHA is any harder to keep up to date than a Maven/Clojars dependency? You still have to go look at the project and see what's changed... #2018-08-27 02:33:27 steveb8n fair point. I agree. I guess the exception is snapshots but I generally don’t use them anyway. also the clojars build/badge provides the latest without a readme update I believe. that’s one way that clojars beats shas for readme maintenance. anyway I’m good and I’ll check latest shas from now on. thanks all #2018-08-27 04:34:04 levitanong What’s the difference between putting some code in "-e '(foo)'" and putting the same code in some namespace with -main, and putting it in an alias main-opt? #2018-08-27 05:57:30 steveb8n @alexmiller interesting turn of phrase. I might “borrow” that #2018-08-28 18:35:42 richiardiandrea @seancorfield question about clj-new's generators - how do I handle the case where I want to modify a file in the project root, like a deps.edn? #2018-08-28 18:35:55 richiardiandrea (and is it common/good practice?) #2018-08-28 18:45:14 seancorfield @richiardiandrea That sounds like a nice enhancement -- to expose the prefix argument somehow. Feel free to open an issue. #2018-08-28 18:45:23 richiardiandrea ok #2018-08-28 18:47:46 seancorfield The main obstacle right now is that you can't specify the prefix (or any other parameters to generate-code under the hood), mostly because the command line processing isn't very sophisticated. Hence the open issue about looking at tools.cli for some of this. #2018-08-28 19:34:48 richiardiandrea @seancorfield sorry other question, can I build a template that accepts a single path in clj-new ? #2018-08-28 19:40:01 seancorfield I'm not quite sure what you're asking there? #2018-08-28 19:48:39 seancorfield If your template function accepts [name & args] instead of the default [name], then any additional arguments you provides on the command-line are passed in... #2018-08-28 19:49:32 richiardiandrea it seems like I always receive an error directly from clj-new if I pass the project name as my-template-test for instance #2018-08-28 19:50:32 seancorfield $ clj -Sdeps '{:deps {clj-new {:git/url "" :sha "108f27159501f8ce12564b398b0ea50def3892b1"}}}' -m clj-new.create template myt/thing
...
$ vi thing/src/clj/new/thing.clj # add & args and print them out
$ clj -Sdeps '{:deps {clj-new {:git/url "" :sha "108f27159501f8ce12564b398b0ea50def3892b1"} my/thing {:local/root "./thing"}}}' -m clj-new.create thing another.generated.thing with args
Generating fresh 'clj new' thing project.
args= (with args)
$ #2018-08-28 19:50:43 seancorfield clj-new does not accept a single-segment name #2018-08-28 19:50:55 richiardiandrea oh ok that's what I feared #2018-08-28 19:50:59 seancorfield It must either be a multi.segment.name or a qualified/name #2018-08-28 19:51:21 seancorfield Phil always regretted allowing that bad practice for lein new 🙂 #2018-08-28 19:51:31 richiardiandrea oh ok got it #2018-08-28 19:51:45 seancorfield You can always say my-template-test.core to get the Leiningen behavior. #2018-08-28 19:51:53 richiardiandrea too bad because it feels natural for JS devs (which are my audience now) #2018-08-28 19:52:02 richiardiandrea yeah #2018-08-28 19:52:09 richiardiandrea have to explain that 😄 #2018-08-28 19:53:12 seancorfield The docs say:
> The project name should be a qualified Clojure symbol, where the first part is typically your GitHub account name or your organization's domain reversed, e.g., com.acme, and the second part is the "local" name for your project (and is used as the name of the folder in which the project is created). An alternative is to use a multi-segment project name, such as foo.bar (the folder created will be called foo.bar and will contain src/foo/bar.clj). #2018-08-28 19:53:57 seancorfield BTW, note above how I generated a template and then used it via :local/root 🙂 Gotta love the whole clj / deps.edn thing! #2018-08-28 19:55:17 richiardiandrea yeah I am using the :local/root thing, it is a nice thing #2018-08-28 19:56:00 richiardiandrea the thing is, JS users do not care/are used to the Java wait of packaging - therefore they consider it an additional burden.. #2018-08-28 19:56:17 richiardiandrea some people here at least 😄 #2018-08-28 19:56:35 richiardiandrea so the naming now for them feels Java-y #2018-08-28 19:56:56 seancorfield JS people should learn the benefits of globally unique names 🙂 #2018-08-28 19:57:15 seancorfield I have zero sympathy! :rolling_on_the_floor_laughing: #2018-08-28 19:58:01 richiardiandrea well, they have a group as well but very few people use it #2018-08-28 20:01:45 seancorfield I will admit that when generating a new template, the qualified/multi-segment thing doesn't work as expected. If you use a qualified/name then the qualified part is ignored. If you use a multi.segment.name you get an illegal project (src/clj/new/multi.segment.name.clj). That's a bug that I haven't opened an issue for yet because there are restrictions on how template functions are named, including their namespace, so I'm not sure how best to resolve that conflict. #2018-08-28 20:02:14 seancorfield The simplest solution would be to allow simple names for templates but... ¯\(ツ)/¯ #2018-08-28 20:10:34 richiardiandrea also I don't see any util to get only the myproj from something like my.long.group/myproj #2018-08-28 20:14:00 seancorfield If you generate an actual project for my.long.group/myproj the qualifier part goes into the namespaces: (! 978)-> clj -Sdeps '{:deps {clj-new {:git/url "" :sha "108f27159501f8ce12564b398b0ea50def3892b1"}}}' -m clj-new.create app my.long.group/myproj
Generating a project called myproj based on the 'app' template.
Tue Aug 28 13:13:34
(sean)-(jobs:0)-(~/clojure)
(! 979)-> tree myproj
myproj
|____.gitignore
|____.hgignore
|____CHANGELOG.md
|____deps.edn
|____doc
| |____intro.md
|____LICENSE
|____README.md
|____resources
| |____.keep
|____src
| |____my
| | |____long
| | | |____group
| | | | |____myproj.clj
|____test
| |____my
| | |____long
| | | |____group
| | | | |____myproj_test.clj
#2018-08-28 21:52:06 richiardiandrea uhm this does not seem to work in my template...I always get: Could not create directory /home/arichiardi/tmp/my.long.group/myproj. Maybe it already exists? Use -f / --force to overwrite it..
Super weird I am doing the same thing that your app is doing #2018-08-28 22:50:52 richiardiandrea oh found the problem, :name is actually used by ->files! didn't expect that 😄 #2018-08-28 20:14:29 seancorfield That's how Leiningen and Boot work. #2018-08-28 20:14:40 richiardiandrea yes that works but I would like to have {:project-name ...} in my data #2018-08-28 20:14:47 richiardiandrea working on a patch #2018-08-28 20:15:09 seancorfield {:project-name ...} in what data? #2018-08-28 20:16:00 richiardiandrea the template data #2018-08-28 20:16:17 richiardiandrea together with {:year ... :name ...} #2018-08-28 20:16:24 seancorfield Ah, yes, the template generator could do with a lot of enhancements. #2018-08-28 20:16:54 richiardiandrea this is something I really need, so patch will be coming #2018-08-28 20:17:00 seancorfield The app generator has :name for (project-name name) #2018-08-28 20:17:30 seancorfield You might consider sending a similar patch to boot-new since it's pretty much the same code, as I recall? #2018-08-28 20:18:16 seancorfield The template already has :raw-name by the way -- which is the whole name. #2018-08-28 20:20:38 richiardiandrea kk will see what I can do #2018-08-28 20:20:45 richiardiandrea maybe I can just roll my own template #2018-08-28 23:32:04 cfleming Am I correct in thinking that there’s no way right now to get the CLI to output the actual Java command that would be executed? #2018-08-28 23:32:24 cfleming Is that cached, or is it only the classpath that’s cached? #2018-08-28 23:58:08 cfleming Never mind, poking through the code, I see the .libs, .cp, .jvm and .main files. #2018-08-29 02:56:02 alexmiller You’re correct in that there is no method to dump the actual full Java command #2018-08-29 02:56:22 alexmiller But it’s assembled from those pieces #2018-08-30 01:34:41 cfleming I don’t really understand the rationale for having separate alias types, e.g. R, C, O, M #2018-08-30 01:35:08 cfleming Why not just have a single alias type and merge in whatever it contains? What’s the use case for separating them? #2018-08-30 02:06:25 kenny @cfleming I find it useful to have a :test alias that includes test dependencies and :main-opts, which I can optionally enable to actually run the tests. #2018-08-30 02:16:00 alexmiller you may have heard that Rich likes taking things apart #2018-08-30 02:51:08 cfleming @kenny But if you had a single alias for that that would work anyway. #2018-08-30 15:40:47 kenny Ah yes, you're right. That simplifies things. I have no use case for the others now. #2018-08-30 02:51:45 cfleming The dependencies would get used when calculation the classpath, and :main-opts would get picked up when running. #2018-08-30 02:53:03 cfleming You don’t need the concept of separation for that. #2018-08-30 02:53:53 cfleming > you may have heard that Rich likes taking things apart
Sure, but I hope he also has reasons for that. I haven’t seen any for this particular case, which is why I asked. #2018-08-30 02:54:59 alexmiller the idea is that they can be used for different parts of the lifecycle and you can choose which part you’re affecting. Or you could say R,C,O,M are the simple part and A is the easy part #2018-08-30 02:55:48 cfleming But surely the different parts of the lifecycle will just get picked up as needed anyway? #2018-08-30 02:56:13 cfleming In Kenny’s test example, the :main-opts will just get ignored when building a classpath. #2018-08-30 02:56:43 cfleming It seems like you’d only use that when you want to remove specific fields when applying an alias, and I can’t imagine a case where that would be useful. #2018-08-30 02:56:51 alexmiller building a classpath involves both the R and C parts #2018-08-30 02:57:35 alexmiller this is not a part of the design that I feel strongly about and I’m not going to give you an impassioned defense for it #2018-08-30 02:58:04 alexmiller as you know, sometimes decisions that seem important early seem less so later #2018-08-30 02:58:51 cfleming Ok, thanks. #2018-08-30 02:59:16 alexmiller fwiw, I only use A these days #2018-08-30 03:00:00 cfleming Yeah, that’s what I would do too. I’m mostly wondering how much support I should be adding for this to Cursive, or if I should just assume everything is implicitly A #2018-08-30 03:00:11 cfleming I think I’ll do that until someone complains about it. #2018-08-30 03:00:15 alexmiller I think that would get you far #2018-09-04 08:42:42 pesterhazy Is there a better way to get a new snapshot build than rm -rf .cpcache ~/.m2/repository/com/bhauman? #2018-09-04 09:19:39 delaguardo clj -Sforce should ignore .cpcache But why you want to delete your local maven repo? #2018-09-04 09:20:04 delaguardo just curious #2018-09-04 09:29:38 dominicm @pesterhazy @delaguardo I'm not certain, but I believe there is delegation to maven for this. So the usual 24h period is in place, regardless of ignoring the cpcache. #2018-09-04 09:49:55 pesterhazy @delaguardo when a new version gets uploaded to maven, I need to download it. By deleting the local cache I make sure the files get re-downloaded #2018-09-04 09:50:11 pesterhazy Just deleting .cpcache won't be sufficient AFAIK #2018-09-04 09:54:20 dominicm @pesterhazy j/w, do boot/lein/mvn provide options for this? #2018-09-04 09:55:01 delaguardo hm… you can not upload to maven new release version (without -SNAPSHOT suffix) so when maven coordinate is changed in deps.edn (e.g. 1.2.0 -> 1.2.1) it will download it. For snapshoted versions it works differently: -SNAPSHOT suffix is replaced with the timestamp and after uploading new snapshot in maven you should have two with different timestamps. #2018-09-04 09:55:52 delaguardo all those cases already handled by tools.deps and there is no need to drop your local cache #2018-09-04 09:56:39 pesterhazy I was clearly talking about snapshot builds in my original question, no? #2018-09-04 09:57:39 pesterhazy let me check the timestamp theory #2018-09-04 09:58:56 delaguardo just try with -Sforce without deleting .m2 #2018-09-04 09:59:04 pesterhazy I will, thanks! #2018-09-04 09:59:53 pesterhazy Pretty sure that boot supports boot -u to re-download new snapshots #2018-09-04 10:00:31 pesterhazy Wanted to check, but the web page doesn't list the command line arguments - it refers you to boot -h - and installing boot takes 5 min #2018-09-04 10:00:35 delaguardo boot -u is for upgrading boot tool itself #2018-09-04 10:00:37 pesterhazy Ok, it's boot -U #2018-09-04 10:00:47 pesterhazy oh no #2018-09-04 10:00:59 delaguardo same, but for the latest snapshot #2018-09-04 10:01:53 pesterhazy looks like I misremembered #2018-09-04 10:49:58 alexmiller tools.deps uses daily update strategy for SNAPSHOTs currently so won’t update new ones even with -Sforce #2018-09-04 10:50:28 alexmiller I’m planning to change that but haven’t gotten to it yet #2018-09-04 10:52:20 alexmiller So for the moment you would need to both delete this portion of your .m2 and use -Sforce (no need to drop ALL of your .m2 though) #2018-09-04 10:54:05 alexmiller Or you could gen a pom with -Spom and use mvn -U dependency:tree to have Maven do it (followed by -Sforce to tell deps to update) #2018-09-04 14:36:40 pesterhazy thanks for the explanation! #2018-09-04 14:37:04 pesterhazy I guess rm -rf is easy enough, it's just hard to explain to others in the team #2018-09-04 15:45:18 alexmiller Yeah #2018-09-05 08:53:49 maleghast Hey there, everyone... I am a little lost, but probably just being stupid; I want to create an uberjar from my tools-deps based application. Any clues, links, etc..? (My Google-Fu has failed me completely) #2018-09-05 09:07:02 maleghast Thanks @dominicm #2018-09-05 09:11:04 maleghast @dominicm - Which Jar approach would you recommend via Pack for Docker-based deployment..? #2018-09-05 09:12:43 dominicm @maleghast I recommend capsule as a good default 🙂 #2018-09-05 09:12:52 maleghast 🙂 #2018-09-05 09:12:54 maleghast Thx #2018-09-05 09:49:13 maleghast @dominicm - I may be doing something stupid (very likely), but on running the following:
clj -A:pack mach.pack.alpha.capsule foundation.jar -e build-dir --application-id foundation --application-version "$(git describe)" -m foundation.main
I can find the file -> mach.pack.alpha.capsule <- but not my jarfile... 😞 #2018-09-05 10:53:10 dominicm What output are you getting? None? #2018-09-05 10:56:32 maleghast er, hold on... #2018-09-05 11:29:56 dominicm There's a file named after the namespace, odd! #2018-09-05 11:30:02 dominicm What's in it? #2018-09-05 11:30:11 dominicm Also, how does your pack alias look? #2018-09-05 11:31:22 maleghast huh? #2018-09-05 11:32:04 dominicm There's a file named mach.pack.alpha.capsule in that folder #2018-09-05 11:32:24 maleghast Yeah, I know #2018-09-05 11:33:27 dominicm What is it? #2018-09-05 11:33:51 maleghast I copied your command from the Pack Github page and just changed the name of the output jarfile and the app main namespace (it's up there ^^) #2018-09-05 11:34:09 maleghast I have no idea what it is, I thought it was supposed to be there. #2018-09-05 11:34:29 dominicm Nope #2018-09-05 11:34:46 dominicm What does your pack section in deps.Edn look like? #2018-09-05 11:36:01 maleghast Er, one sec #2018-09-05 11:36:34 maleghast :pack {:extra-deps
{pack/pack.alpha
{:git/url ""
:sha "1d455f6c591f440560c57a29e6d930004d735dcf"}}
:main-opts ["-m" "mach.pack.alpha.capsule"
"-m" "foundation.main"]}
#2018-09-05 12:35:53 mike_ananev @maleghast I recommend you to stay with lein for some time. pack.alpha, depstar and other have some lacks... one of them copy src files into uberjar, another doesn't copy resource folder into uberjar. I've investigated all of them and IMO lein is still preferable way to build uberjar. #2018-09-05 13:29:46 dominicm pack does put src files in, because that's what you generally want. If you want to AOT, that is supported with pack, but you do it as an earlier step.
I'm happy to point you at how you can do this, and we're working on improving the documentation to make it much clearer on how to do this kind of thing. #2018-09-06 09:17:18 Andreas Liljeqvist please point 🙂 #2018-09-05 13:00:28 maleghast Thanks @mike1452 I will take a look... The app I am trying to UberJar is built on top of a library that @dominicm works on, that uses tools.clojure and deps.edn and as such uses neither leiningen or boot, and I am loath to add either into the mix if I don't have to. That being said, if I have to... #2018-09-05 13:21:00 maleghast @dominicm - I think that I know what's wrong with the command I pasted above... #2018-09-05 13:21:25 maleghast this works:
clj -A:pack foundation.jar --application-id foundation --application-version "$(git describe)" -m foundation.main
#2018-09-05 13:27:36 maleghast @dominicm - yeah, that does work... However I need to figure out how to package up the CSS etc. as the app is functioning, but looks AWFUL 😉 #2018-09-05 13:27:41 dominicm @maleghast yeah, I figured that the alias you had already had the mach.pack.alpha.capsule in the alias 🙂 #2018-09-05 13:27:58 maleghast Yeah, that's what I worked out as well... #2018-09-05 13:28:20 dominicm @maleghast the -e build_dir option is a flag to provide an extra directory to be combined into the jar. So you can build your css/js into a directory, and have it run. #2018-09-05 13:28:31 dominicm If you're using edge, there's an uberjar script in master which has these steps in 🙂 #2018-09-05 13:28:42 maleghast Ah... Thanks! 🙂 #2018-09-05 13:28:51 maleghast (I am using edge) #2018-09-05 14:56:43 pesterhazy #2018-09-05 14:58:55 pesterhazy e.g. clojure -Spath | unpack-cp, then cd to that directory and run grep from there #2018-09-05 18:17:23 mike_ananev @dominicm Did you use pack/pack.alpha? can't understand how to avoid copying source files into uberjar using this tool #2018-09-05 18:39:45 dominicm I wrote it. Is having source files in there bad? #2018-09-07 10:08:40 mike_ananev Well it depends. I think it should as parameter, like lein does ( :omit-source true) . When I put my uberjar in external untrusted environment I don't want that my source files be availabale for other people. #2018-09-07 10:09:56 dominicm I haven't looked into that kind of use case particularly, but I understand now. Skipping the :paths seems like a possible flag that can be added though, it shouldn't be too difficult. #2018-09-07 19:03:48 mike_ananev Yeah. It would be great. #2018-09-06 21:37:12 gas how do I clear the cache trying to do clj -A:new figwheel-main hello-world.app --reagent but getting yesterdays cached version of fighweel.main template? #2018-09-06 22:08:05 bhauman @gas 0.1.9 is out so you can just update to that #2018-09-06 22:27:40 gas yeah I see that however also saw you updated template so tryed to run clj -Anew but keep getting old template settings? #2018-09-06 23:29:48 seancorfield @gas does clj -Sforce work to clear the cache (that clj keeps)? #2018-09-06 23:31:05 seancorfield (or you can just delete the .cpcache folder where you are running clj and that should force it to recreate any classpaths) #2018-09-06 23:31:15 seancorfield I think Alex mentioned a bug around snapshot handling... #2018-09-06 23:31:30 gas trying -Sforce @mo where is cpcache udner home? #2018-09-06 23:32:15 seancorfield The .cpcache folder exists wherever you run clj #2018-09-06 23:32:28 seancorfield (it creates that in every folder that you run it in) #2018-09-06 23:33:17 gas ok -Sforce didnt work. #2018-09-06 23:33:40 seancorfield Per Alex's comment? #2018-09-06 23:33:51 seancorfield (snapshots only get checked for once a day) #2018-09-06 23:34:51 gas yeah but not sure how bhaumann figwheel-main template works as it doesnt seem to have a version. presume its goes off git? #2018-09-06 23:36:32 gas dont seem to have .cpcache where I am running clj? on a mac not in home wither. hmm found it in .clojure/.cpcache #2018-09-06 23:37:46 gas delete ~/.clojure/.cpcache ?? #2018-09-06 23:37:56 bhauman templates are a bit different #2018-09-06 23:38:26 bhauman I’m releasing a new template very soon #2018-09-06 23:39:28 seancorfield clj-new requests RELEASE or SNAPSHOT versions as part of the dependency lookup as I recall... Let me go look at the code. It's been a while since I wrote it. #2018-09-06 23:39:48 bhauman oh that would be cool #2018-09-06 23:40:45 gas @bhauman ok - I have copied your template and works great - FYI only issue in template at moment you need react as deps for reagent config for it to work with regent 0.8.1 #2018-09-06 23:42:05 seancorfield OK, so here's what happens in clj-new... #2018-09-06 23:42:31 seancorfield ...it looks for <template>/clj-template then <template>/boot-template then <template>/lein-template, in that order. #2018-09-06 23:43:40 bhauman @gas weird, I tested every config, I’m surprised #2018-09-06 23:44:02 bhauman @seancorfield so no snapshot versions? #2018-09-06 23:44:07 seancorfield For the clj-template, if the template name looks like a git URL or a filesystem local path, it will only use that. #2018-09-06 23:44:31 bhauman oh well thats good #2018-09-06 23:46:06 seancorfield Otherwise it will either request a RELEASE version or a SNAPSHOT version -- but I'm checking whether the flag to expose snapshot is in place... #2018-09-06 23:46:32 bhauman @gas it shouldn’t need react at at all #2018-09-06 23:46:33 seancorfield ...right now, that flag isn't exposed in clj-new (only in boot-new ... and lein new) #2018-09-06 23:47:15 bhauman very good to know #2018-09-06 23:48:00 seancorfield BTW, a "git template" looks like <url to repo>@<SHA> and the last part of the URL is considered to be the template name. #2018-09-06 23:48:13 seancorfield So you can pull specific versions of templates down from GitHub using that. #2018-09-06 23:48:30 gas @bhauman ok I commented out react and it works this time! Only diff is i had the #framework part in my core.cljs but dont see any react stuff in that code #2018-09-06 23:50:38 bhauman @seancorfield the snapshot part won’t work? #2018-09-06 23:51:13 seancorfield The git dependency won't work. #2018-09-06 23:52:01 seancorfield clj -A:new hello-world.app #2018-09-06 23:52:28 seancorfield That would work if that was indeed the name of the repo -- but you have -template on it. #2018-09-06 23:53:15 bhauman gotcha #2018-09-06 23:53:27 bhauman there’s already a figwheel-main repo 🙂 #2018-09-06 23:53:31 seancorfield I could provide an option to override the actual template name (like I do with local paths: /path/to/some/code::my-template #2018-09-06 23:53:49 bhauman I think the git url covers this case pretty well #2018-09-06 23:54:15 seancorfield Well, the problem is that the template name doesn't match the repo path. #2018-09-06 23:54:44 bhauman oh #2018-09-06 23:54:46 bhauman ok #2018-09-06 23:54:57 bhauman hmmmmm #2018-09-06 23:55:06 seancorfield If I added that option, you could do clj -A:new hello-world.app
#2018-09-06 23:55:41 seancorfield I should make it consistent on both git and local deps really... #2018-09-06 23:56:09 seancorfield And figure out a way to enable snapshot lookup... #2018-09-06 23:59:37 seancorfield I'll work on those tomorrow afternoon or over the weekend. #2018-09-07 07:43:57 levitanong Hi guys, does anyone have a cljs tools-deps library I can look at? Trying my hand at it, but somehow the library can’t see itself. Lol #2018-09-07 11:36:52 dominicm I'm interested in solving TDEPS-12, @alexmiller have you thought about how you'd like it to be done? #2018-09-07 15:48:35 isaac how to use clj cli compile java code #2018-09-07 15:48:54 isaac I have some java code in classpath #2018-09-07 16:13:21 alexmiller @dominicm I don’t think anything on that ticket is very good thinking about this problem #2018-09-07 16:14:04 alexmiller the problem is that the assumption going into the design is that there is a 1-1 correspondence between library name and artifact #2018-09-07 16:14:43 alexmiller this is a case where that is not true #2018-09-07 16:15:14 alexmiller everything else flows from that and without addressing that fundamental issue, there is no way to fix the issue #2018-09-07 16:16:13 alexmiller I’ve been thinking about this for a long time and I think you have to solve the “name” part or you’re not fixing anything #2018-09-07 16:19:03 alexmiller the library name here is used for dual purposes - identification in tools.deps, and lookup in Maven (the groupId:artifactId). In many places we key maps based on #1 so that really needs a new dimension of context that is more than just #2. #2018-09-07 16:19:48 alexmiller that means encoding the classifier into the name (so you get uniqueness of identity) and have the Maven extension know how to decode that for lookup #2018-09-07 16:20:10 alexmiller that could be done with a name encoding, or by using something other than a symbol as the library name #2018-09-07 16:22:39 dominicm Interesting. When discussing with @malcolmsparks I dismissed the notion of using a vector or some encoding for the key.
Something not mentioned is that there's more than just classifier as a dimension in Maven, there's also type. I suspect encoding is a bad solution because it's not information dense enough. #2018-09-07 16:24:11 alexmiller #2018-09-07 16:32:03 dominicm Sure. But then you have type too. #2018-09-07 16:32:35 alexmiller type is not important #2018-09-07 16:32:42 alexmiller tools.deps is about classpaths #2018-09-07 16:32:54 alexmiller it is not about “providing a generic interface to all things Maven” #2018-09-07 16:33:30 alexmiller the only type we care about is jars #2018-09-07 16:34:43 dominicm Are we certain type isn't important? If so I agree that the symbol is the only important one.
It would be a shame to use encoding, and then end up having to cram something additional in. #2018-09-07 16:36:09 alexmiller give me an example where type is important #2018-09-07 16:36:32 alexmiller (I’ve done this exercise and not come up with anything) #2018-09-07 16:37:04 alexmiller another option is to support an expanded lib name like {:groupId "group.id", :artifactId "artifactId", :classifier "classifier"} #2018-09-07 16:37:52 alexmiller doing one does not preclude doing the other later #2018-09-07 16:39:20 dominicm I don't have one to hand, that's certain. In that case I will defer to your data. My objection was weak and I was erring on the side of exposing everything in the case of not having access to any data.
#2018-09-07 16:40:04 alexmiller tools.deps has been out for about a year and I haven’t seen a request or question about anything other than classifiers #2018-09-07 16:40:05 dominicm If we did that, would it be a consistent return value? I'm guessing it would involve changing the current lib map spec? #2018-09-07 16:40:22 alexmiller what is “it” in above? #2018-09-07 16:40:47 dominicm The map version of the lib name. #2018-09-07 16:41:26 alexmiller yes, would change (grow) the spec and a couple things in the Maven extension canonicalization #2018-09-07 16:41:49 alexmiller I really need to talk to Rich to make a decision on that #2018-09-07 16:43:24 dominicm At the moment I rely on it being a symbol in pack, in order to perform naming. That seems like a breaking change for me. Am I relying on a property I shouldn't? #2018-09-07 16:43:38 alexmiller the code changes are pretty trivial. thinking clearly about the problem here is the important part. #2018-09-07 16:52:37 dominicm > choose whatever name you want with lowercase letters and no strange symbols.
I was just looking into which character was appropriate. Maven is surprisingly coy about what is allowed... #2018-09-07 16:54:27 dominicm I guess I will just have to test. #2018-09-07 17:28:43 lilactown lol so I blew out ~/.m2 and now when I try and run clj:Error: Could not find or load main class clojure.main
#2018-09-07 17:44:32 dominicm @lilactown I'm guessing because the classpath is cached #2018-09-07 17:48:02 alexmiller yes, if you manually muck with .m2, you’ll need to use clj -Sforce #2018-09-07 17:48:27 alexmiller currently clj does not do any checks to verify that all the dirs/files on the classpath exist #2018-09-07 17:49:37 alexmiller I’m a bit on the fence about it - that could be added, but has the downside of a) adding startup time and b) not being tolerant of (for example), a missing src path on the cp. although I guess you could maybe only check the files. #2018-09-07 18:30:49 zentrope Maybe just a warning to “if in doubt, clj -Sforce” on “missing” errors? #2018-09-07 18:32:53 lilactown yeah, I didn’t know about -Sforce. that would be a helpful enhancement #2018-09-07 18:38:37 zentrope Or, could do the path checking AFTER the error occurs, in which chase “start up time” isn’t a factor. #2018-09-07 18:40:26 zentrope Verify that there are missing paths for a good error message, not re-do the whole op. #2018-09-07 19:56:12 alexmiller the benefit of doing it before is that you can potentially fix it, not error #2018-09-07 20:00:17 alexmiller if stuff is stale, then force #2018-09-07 20:01:27 alexmiller the question I’d like to answer is - how long does it take to do the check for a reasonable size classpath #2018-09-07 23:35:08 zentrope Right. The “after” idea is to keep the speed benefits, but provide a clearer option for the user to try: same as it is now, just slightly clearer. #2018-09-08 07:24:06 malcolmsparks @alexmiller @dominicm One example where type is important is where a project might depend on a .war file, containing some functionality. See https://codesjava.com/eclipse-maven-java-web-project for example. This is more likely to happen inside a large enterprise using J2EE, I doubt you'd find an example in 'public' repos but there are vast private ones out there. This still is about the classpath (Maven itself is primarily about classpaths, not file distribution). There are other types too: https://docs.oracle.com/cd/E19830-01/819-4712/ablgz/index.html. I don't think it's worth adding support for them in tools.alpha, but at the same time it would be good if any design around supporting classifiers can be extended (via growing) to support types as well. #2018-09-08 07:30:27 malcolmsparks I think the 1-1 correspondence is actually quite a nice design. If you think of the artifact as the logical entity defined in the pom.xml file (with its version and dependencies), rather than an actual file in the Maven repo. 99% of cases there is a single file for the artifact, but sometimes there are also ancillary files (one per classifier), sometimes called secondary artifacts. While Maven does support depending on a primary artifact file of, say, version 1.0, and a secondary artifact of a different version, say, version 1.1, this is such a questionable practice it is unlikely to be worth supporting. #2018-09-08 07:33:24 malcolmsparks So in conclusion, I don't think there needs to be an extended lib name of {:groupId "group.id", :artifactId "artifactId", :classifier "classifier"}. The libname, with the :mvn/version, already uniquely addresses the logical artefact (defined in its pom). There should be a way of specifying the required classifiers, perhaps with a :mvn/classifiers entry. #2018-09-08 07:37:54 malcolmsparks One minor point is that Maven makes you a declare a dependency on each artifact file you want to download (and you just normally repeat the GAV coord with a different classifier). Therefore there might need a way of saying "this classifier please, but include the primary artefact too", and also 'this classifier please, but not the primary artefact". I think the latter case could be supported later if required via growing the design. #2018-09-08 12:39:33 alexmiller I can give you at least one more example that has nothing to do with native stuff #2018-09-08 12:41:31 alexmiller ClojureScript relies on a few contrib libs (like tools.reader). All of hose libs get built in both a source form (and deployed with no classifier) and in aot form (deployed with :aot classifier). #2018-09-08 14:53:44 alexmiller the idea of pushing this into the coord is interesting. The current extension SPI already supports extensions returning multiple artifacts per lib. I do wonder about whether this has impacts for walking the transitive deps? Something to look at. #2018-09-08 14:56:00 alexmiller I guess there’s still only one pom defining deps for a lib, even in the case where there are multiple classifiers #2018-09-08 16:06:31 alexmiller as I looked at it more, I like this solution and I have implemented it and committed the change for the next release - see TDEPS-12 for more #2018-09-08 17:54:17 gfredericks I was thinking again about self-contained clojure files
I realized you could have a pretty clean impl at the cost of a mildly longer shebang line, if you publish a library (ideally with a very short name) that can read deps out of lines 2->N of a file, and execute everything after that using -Sdeps
so the shebang would be roughly #! clojure -Sdeps '{:deps {lib7 {:mvn/version,"1"}}}' -m lib7.main
I think the tricky part is that lib7 couldn't do a unix exec to replace the original jvm; so maybe the cleanest thing for it to do would be to have tools.deps as a dep, and call it internally, and make its own classloader with the resulting classpath...
so
A) are there any libs yet that accomplish anything comparable to this?
B) is this crazy or impossible? #2018-09-08 17:59:29 gfredericks Startup time would be interesting #2018-09-08 18:29:04 dominicm There's a library to make isolated class loaders from a deps edn. #2018-09-08 18:29:20 dominicm Startup time would be boot-like #2018-09-08 18:31:28 gfredericks do you recall the library's name? #2018-09-08 18:38:53 dottedmag @gfredericks Shebangs do not get parsed into arguments. #2018-09-08 18:39:15 dottedmag And yes, someone has done something like that. #2018-09-08 18:40:09 dottedmag IIRC via file that can be read either as a shell script (that runs clj) or as Clojure code (where shell header gets ignored via e.g. ;). With #!/bin/sh shebang. #2018-09-08 18:41:19 gfredericks oh if you can't in any syntax pass two args to clojure then I guess that would kill that idea #2018-09-08 18:41:47 dottedmag #!/bin/sh
; exec clj .... $0
<clojure code follows>
#2018-09-08 18:42:10 dottedmag Something like this. #2018-09-08 18:42:27 gfredericks having lots of deps crammed into one line is unfortunate #2018-09-08 18:43:11 gfredericks I bet some fancy bash could support multiple lines #2018-09-08 18:43:24 dottedmag Multiple lines of what? #2018-09-08 18:43:37 gfredericks deps #2018-09-08 18:44:07 dottedmag Why? I don't remember a requirement that shell script has to fit on a punched card 🙂 #2018-09-08 18:44:19 gfredericks just annoying when viewing and editing #2018-09-08 18:44:23 gfredericks not a deal breaker #2018-09-08 18:44:33 dottedmag I'd generate this script instead of editing it manually. #2018-09-08 18:44:47 gfredericks generate it from what? #2018-09-08 18:45:07 gfredericks I guess I mean the thing I'm going for is casual self-contained shell scripts #2018-09-08 18:45:13 gfredericks not tied to anything else #2018-09-08 18:45:17 gfredericks so generating it kind of defeats the purpose #2018-09-08 18:45:18 dottedmag and yes, should be trivial: #!/bin/sh
; CLJ="clj ..."
; CLJ="$CLJ ..."
; CLJ="$CLJ ..."
; exec $CLJ
<clojure code follows>
#2018-09-08 18:46:05 gfredericks I wonder if you could take advantage of the deps map being a noop to have it outside the comments #2018-09-08 18:46:11 gfredericks so you get syntax highlighting and editor help #2018-09-08 18:48:57 gfredericks I get Syntax error: ";" unexpected for that sort of thing #2018-09-08 18:55:30 gfredericks this is the holy grail I think
#!/bin/sh
... magic bash
{:deps {...
...
...
...}}
... more magic bash, with exec
(ns wahoo ...)
#2018-09-08 19:46:52 borkdude Does anyone have an example of how to use clj/tools-deps with AOT + uberjar? #2018-09-08 19:47:09 gfredericks ":"; seems designed to avoid the error I mentioned above #2018-09-08 19:48:15 borkdude yes, it was the shortest bash command I could find that was also valid clojure #2018-09-08 19:48:22 borkdude I think @mfikes came up with it #2018-09-08 19:51:03 gfredericks would "" not work? #2018-09-08 19:51:12 gfredericks I guess not #2018-09-08 19:51:20 gfredericks what does : mean in bash? #2018-09-12 12:42:50 souenzzo it's not plain true
it's a (def : (constantly true)) #2018-09-08 19:51:57 borkdude it’s an alias for true #2018-09-08 19:52:05 gfredericks o_O #2018-09-08 19:52:17 gfredericks sure why not #2018-09-08 19:53:19 gfredericks so your AOT/uberjar question is about asking how to take a src dir and AOT compile it and put that in an uberjar? #2018-09-08 19:54:38 borkdude yes, with tools.deps. With lein I can find docs on that, but with tools.deps not so much I guess #2018-09-08 20:10:34 alexmiller There are some things there to do it #2018-09-08 20:11:04 borkdude Thanks! #2018-09-08 19:54:58 gfredericks I thought I saw an uberjaring lib at some point #2018-09-08 19:55:27 gfredericks w.r.t. AOT, it might be true that by calling compile enough you will get class files written to the classes dir #2018-09-08 19:55:34 gfredericks but there might be lots of edge cases involved that I don't know about #2018-09-08 19:56:09 gfredericks e.g., if you have a top-level namespace that requires everything else, try clojure -e '(compile (quote that.ns))' #2018-09-08 20:01:49 borkdude if [ ! : ]; then
echo "True!"
else
echo "False!"
fi
# prints "False!"
#2018-09-08 20:07:02 gfredericks who wouldn't want to be able to write code like that! #2018-09-08 20:10:10 borkdude however, this doesn’t seem to work:
if [ : && : ]; then
echo "True!"
else
echo "False!"
fi
#2018-09-09 11:52:31 gfredericks I figured it out!
#!/usr/bin/env bash
":"; DEPS=\
'{:deps
{org.apache.commons/commons-compress {:mvn/version "1.17"}}}'
":"; exec clojure -Sdeps "$DEPS" "$0"
#2018-09-10 13:02:34 rickmoynihan Neat. I’ve had a need for something like this this morning.
Have been meaning to ask why clojure doesn’t barf on the #!/usr/bin/env bash? #2018-09-10 13:04:33 rickmoynihan (read-string "#!/usr/bin/env bash") blows up… but clojure seems to somehow let it through… #2018-09-10 13:38:56 gfredericks #! is indistinguishable from ; AFAIK
your read-string call fails with ; as well, because there's nothing for it to return -- add a second line with a valid form and both will work #2018-09-10 13:39:25 rickmoynihan Yeah scroll down #tools-deps 🙂 #2018-09-10 13:39:34 gfredericks oh sorry #2018-09-10 13:39:52 rickmoynihan not at all; thanks for answering 🙂 #2018-09-09 11:52:52 gfredericks ☝️ that seems sufficient to let you keep your deps in the clojure-syntax part of the file #2018-09-09 12:12:46 gfredericks also an advantage of not putting clojure in the shebang is that you don't have to give the fully qualified path to it #2018-09-09 12:15:37 gfredericks pro-tip: if you ever have to use a lib that contains a ' in its version string (which would threaten the bash quotation), use \u0027 instead #2018-09-09 12:18:59 dominicm You can use a quote for a version? #2018-09-09 12:20:10 gfredericks probably
{:mvn/version "12.43.01'-sure"}
I mean I haven't tried it #2018-09-09 12:20:47 dottedmag {haskell/haskell {:mvn/version "'"} #2018-09-09 14:30:13 borkdude @dominicm cool! #2018-09-09 15:28:50 pesterhazy Weird and wonderful #2018-09-10 12:11:47 aitem Hi to all.
How i can omit source and obfuscate my target .jar file using deps.edn and clojure cli?
Previously i used leiningen and proguard and all works fine.
But in current project we are use clojure cli and juxt/pack.alpha for building .jar. #2018-09-10 13:23:45 rickmoynihan (read-string "#!/usr/bin/env bash") blows up… but clojure seems to be ok with it… This is the only bit I don’t understand, presumably the clj/`clojure` tools strip this out? #2018-09-10 13:29:21 dominicm @rickmoynihan they do not, this is in clojure.core #2018-09-10 13:29:34 rickmoynihan really #2018-09-10 13:29:35 rickmoynihan wow #2018-09-10 13:29:52 dominicm ! is technically a tagged literal which comments out the whole line #2018-09-10 13:30:12 rickmoynihan that was my first thought… but why does read-string blow up? #2018-09-10 13:30:45 rickmoynihan ahhh hold on maybe it’s because there’s no more input… #2018-09-10 13:30:58 rickmoynihan yeah that’s it #2018-09-10 13:30:58 rickmoynihan thanks #2018-09-10 13:33:21 rickmoynihan (read-string "#!/usr/bin/env bash\n 10") works (read-string "#!/usr/bin/env bash") fails #2018-09-10 13:33:23 dominicm boring answer 😉 Why is it always a boring answer #2018-09-10 13:33:33 dominicm (Because physics isn't broken) #2018-09-10 13:40:38 rickmoynihan Though I guess even then this was useful, even then… more so now we have the clj tools #2018-09-10 14:09:20 dominicm This was clearly always the plan 😛 #2018-09-10 15:21:49 alexmiller Rich plays the longest game #2018-09-10 15:36:08 dominicm I'm waiting for an IDE for clojure to appear in the clojure codebase. #2018-09-10 16:20:48 alexmiller No comment #2018-09-10 23:24:05 Joe Lane Oh man this comment is going to keep me up tonight. #2018-09-11 03:05:24 alexmiller Mission accomplished #2018-09-10 15:42:48 sparkofreason I need to use gen-class it implement an abstract Java class, but am having no luck getting it to load in the REPL for cursive. I assume it needs to be AOT'ed to be present on the classpath for clj. How is this accomplished with deps.edn? Or do I need a separate tool? #2018-09-10 16:02:44 sparkofreason If I use compile and add classes to :extra-paths I get this exception on import. #2018-09-10 16:31:21 nicola @aitem i’m curious too how distribute clojure closed source jars. How datomic is distributed? May be @alexmiller help us. #2018-09-10 16:56:31 alexmiller Don’t know - I’m not on the Datomic team :) #2018-09-10 17:57:17 nicola Who can we ask about this? #2018-09-10 18:26:05 alexmiller ask in #datomic #2018-09-10 16:33:26 seancorfield @rickmoynihan Looking at that code makes me wonder what #< is for since, right now, both in EDN and Clojure, the reader for that simply throws an exception. #2018-09-10 16:43:01 rickmoynihan @seancorfield: yes that’s a curious one #2018-09-10 16:43:45 seancorfield I'd forgotten about ## as well -- and #! was new to me too. #2018-09-10 16:43:57 dominicm #<foo> might be something? #2018-09-10 16:44:54 rickmoynihan yeah ## was new in 1.9 #2018-09-10 16:45:04 seancorfield Can you override how #< is read? Or do you mean "reserved for some future, unknown purpose"? #2018-09-10 16:46:41 rickmoynihan I didn’t think the reader was extensible/overridable; at least not beyond tagged literals. #2018-09-10 17:08:18 dominicm @seancorfield I'm guessing a bit. #2018-09-10 17:21:04 mfikes #< is for values that have no way to be read #2018-09-10 19:24:40 andy.fingerhut Wow, I had never noticed that #! special case in the Clojure reader before. I won't say I never saw it, because I am sure I looked at the linked group of source lines at least a dozen times over the years. #2018-09-10 20:42:00 borkdude what’s an example with #<? #2018-09-10 20:48:42 andy.fingerhut As in, is there some object that will print as a sequence of character beginning with #< ? I don't know if there is one today. #2018-09-10 20:52:14 andy.fingerhut Scanning through the Clojure implementation for print and pprint I didn't notice any such case. #2018-09-10 21:10:08 alexmiller Not to my knowledge #2018-09-10 21:10:33 alexmiller I think that’s just reserving space in the reader #2018-09-10 22:49:43 gfredericks it was used for everything prior to #object wasn't it? #2018-09-10 22:58:19 alexmiller You think it’s that old object syntax? #2018-09-10 22:58:32 alexmiller Maybe so #2018-09-10 23:11:23 gfredericks my interpretation was always that #<> was the way unreadable things were printed, and the #< entry in the reader was to produce the "unreadable form" exception whenever you accidentally tried to read those things #2018-09-10 23:14:41 gfredericks here's the reader error messages from 1.2.1:
user=> (-> (Object.) pr-str read-string)
java.lang.RuntimeException: java.lang.Exception: Unreadable form (NO_SOURCE_FILE:0)
user=> (read-string "#,comma")
java.lang.RuntimeException: java.lang.Exception: No dispatch macro for: , (NO_SOURCE_FILE:0)
so I thought of the #< codepath as the way to avoid the less helpful No dispatch macro for: < message you'd get otherwise #2018-09-10 23:15:00 gfredericks and it's still useful today for reading data produced by older clojures #2018-09-11 00:04:19 john I could have sworn I've seen the #<some-object> notation before #2018-09-11 00:29:04 gfredericks $ clojure -Srepro -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.2.1"}}}' -e '(Object.)'
#<Object #2018-09-11 00:39:08 seancorfield Changed in 1.7 Clojure 1.2.1
#<Object
#2018-09-11 00:39:22 seancorfield (the new CLI stuff makes this so easy to test!) #2018-09-11 00:42:11 gfredericks for clojure in 1.2.1 1.3.0 1.4.0 ... #2018-09-11 03:50:05 alexmiller probably related to relative paths if I had to guess #2018-09-11 03:50:17 alexmiller is it a transitive local dep? #2018-09-11 03:52:53 alexmiller there is a known issue that transitive local deps resolve paths in terms of your current directory, not in terms of the dep, which can lead to issues with finding the correct “dep root” (where the deps.edn would be expected) #2018-09-11 03:54:52 richiardiandrea @alexmiller yes it is a transitive dep that of one of the sub projects and one colleague has noticed that if he specifies it top level the problem goes away. #2018-09-11 03:55:25 richiardiandrea Thanks then good to know the problem is known I will check Jira #2018-09-11 03:57:09 richiardiandrea @cfleming thanks this is the reason of the Cursive message ^ #2018-09-11 03:59:42 alexmiller that’s the placeholder for it, there might be another one that’s same thing #2018-09-11 11:54:36 cfleming @richiardiandrea Thanks for letting me know. #2018-09-11 17:08:29 dominicm @alexmiller someone in #leiningen just discovered that openjfx uses <type>pom</type> which wouldn't be supported by deps.edn #2018-09-11 17:08:36 dominicm just as a data point #2018-09-11 17:08:57 alexmiller true #2018-09-11 17:10:38 alexmiller tools.deps is hardcoded to look only for jar extension #2018-09-11 17:10:56 alexmiller the original example in TDEPS-12 is all jars #2018-09-11 17:33:29 dominicm Yep. This was something @malcolmsparks pointed out a little later.
Pom is something that is logically consistent with classpath building. I don't know how type interacts with classifier, I suspect it's multiplicative. #2018-09-11 17:35:20 malcolmsparks Not quite. Each classifier/type combination is specified in a separate dependency element in the POM. #2018-09-11 17:38:25 malcolmsparks @alexmiller thanks for comment in TDEPS-12, I've been able to build a local version of tools.deps and patch it into my /usr/bin/clojure script, no problem. Still testing. #2018-09-11 17:43:59 dominicm It might be that a classifier key is wrong, and it should be named otherwise, and be maps with classifier and type in. #2018-09-11 17:44:03 alexmiller usually I just add the source directory of tools.deps.alpha to the front of the lib path in my clojure script :) #2018-09-11 17:45:45 alexmiller poms are used for determining transitive deps but are not themselves artifacts that get put on the classpath (these are independent parts of the tools.deps extension spi) #2018-09-11 17:46:00 alexmiller all classifiers in a group/artifact/version share the same pom #2018-09-11 17:46:15 alexmiller which is in some ways deeply weird #2018-09-11 17:46:43 alexmiller and really reinforces that GAV should primarily be about “one” artifact #2018-09-11 17:47:36 alexmiller I think you could argue pretty readily that the case in TDEPS-12 for example should be different artifactIds, not classifiers, but I don’t run the world :) #2018-09-11 18:26:32 dominicm I was thinking the same. But I did realize that having structure provides the opportunity for tooling to inspect it. Eg automatically fetching src & javadoc classifiers.
In the TDEPS-12 case, tooling has less of a place, but could feasibly be done by a maven plugin #2018-09-11 18:29:29 alexmiller source and javadoc are secondary info (just like Clojure meta) and make total sense to me #2018-09-11 18:30:33 alexmiller even things like aot as a secondary classifier make sense to me as a variant of the same artifact #2018-09-11 18:31:17 alexmiller any case where you’re putting more than one artifact from the same GAV on your classpath at the same time seems … weird #2018-09-11 18:37:11 dominicm I don't know GAV. Google isn't too revealing either, what is it? #2018-09-11 18:38:09 alexmiller sorry #2018-09-11 18:38:09 alexmiller I’ve merged with the borg apparently #2018-09-11 18:38:30 cfleming Group Artifact Version (standard Maven ID) #2018-09-11 18:38:40 alexmiller groupId/artifactId/version #2018-09-11 18:38:55 alexmiller nice slack, out of order message delivery #2018-09-11 18:39:27 dominicm My guess was surprisingly accurate. The version part didn't quite fit for me. #2018-09-11 18:40:37 dominicm Just so I'm clear, do you think javadoc should be a separate GAV with a pointer from the root? Or that context does make sense for a classifier? #2018-09-11 18:57:18 alexmiller no, I think the existing usage of classifiers for javadoc and source make sense #2018-09-11 18:57:33 alexmiller I think cases like the one in TDEPS-12 are weird #2018-09-11 18:57:50 alexmiller all those classifiers should be separate artifacts #2018-09-11 20:33:54 dominicm I wonder how hard it would be to have a tool which synchronized your Maven deps to add a profile with javadoc and source. I'd love that for stack traces.
A bit of a shame that it would be duplicative to isolate to an alias with the current classifier data shape. Not terrible I suppose though. #2018-09-11 20:37:35 dominicm I suppose you could use such a tool through -Sdeps too, but then cider would have to become aware of it... Unless it was a dependency of cider, then everyone would have it and be happy forever! #2018-09-11 20:48:33 seancorfield Not everyone uses CIDER tho'... #2018-09-11 20:49:18 seancorfield (heresy, I know!) #2018-09-11 20:53:58 dominicm I think intellij is the only editor that doesn't. And atom. #2018-09-11 21:01:50 dominicm Intellij probably already does this, so is less interesting. Atom would miss out, but could integrate directly with the library. #2018-09-11 22:06:36 alexmiller intellij does this already #2018-09-11 22:07:02 alexmiller that is, download and use source and javadoc for libs #2018-09-12 04:12:18 cfleming Right, Cursive does that for lein libs, it doesn’t work for deps yet #2018-09-12 04:43:37 alexmiller ah #2018-09-12 04:45:25 cfleming Clojure actually makes this pretty tricky, so in Cursive it works on demand - if you try to go to the source of a Java class IntelliJ will decompile it and prompt you to download the source at that point, it doesn’t do it up front since it’s basically impossible to know which artifacts have sources (most don’t in Clojure) #2018-09-13 16:28:37 abp When automating tests and builds with clojure we need to download maven dependencies to the project folder ideally, so we can use gitlab CI caching on the maven dependencies in the project folder. Is there any way to do this with the clojure shellscript and tools.deps? With leiningen we used local-repo https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L145 #2018-09-13 16:38:06 alexmiller Yep, use a root attribute in deps.edn of :mvn/local-repo #2018-09-13 16:44:52 abp We just found it too, but thanks anyway 🙂 #2018-09-13 18:25:47 richiardiandrea Hello folks! I am a bit confused by this:
clj -C:gen:new:tmpl -Stree <- no deps is shown
clj -R:gen:new:tmpl -Stree <- deps are shown
but the doc says that -C is for building the classpath... #2018-09-13 18:26:47 richiardiandrea if I am building the classpath, why is -C not showing my deps in -Stree? #2018-09-13 18:27:03 richiardiandrea (and of course I don't have them in the classpath as well) #2018-09-13 18:32:39 hiredman is there a reason you aren't using -A #2018-09-13 18:34:35 hiredman my admittedly not very deep understanding is, -A does a lot of stuff, an the other flags like -C and -R do subparts of what -A, I suspect -C only merges the paths entries not the dependency entries. using -A seems to provide most of the kind of default what you would expect from "merging" #2018-09-13 18:35:43 hiredman there used to be a comment (maybe there still is) the default deps.edn file that said ''' #2018-09-13 18:35:48 hiredman ;; make-classpath aliases (-C) affect the classpath generation, options:
;; :extra-paths - vector of additional paths to add to the classpath
;; :classpath-overrides - map of lib to path that overrides the result of resolving deps
#2018-09-13 18:36:28 richiardiandrea @hiredman my alias contains :main-opts and I do not want to use it for new and gen for now #2018-09-13 18:36:43 richiardiandrea I am launching a REPL against them #2018-09-13 18:37:07 seancorfield Beat me to it... Yeah, @richiardiandrea -C only pulls in paths and the library overrides. -R is for dependencies (`:deps`, :extra-deps, and :override-deps I think). #2018-09-13 18:37:40 richiardiandrea ok I think I missed that part of the doc, thanks a lot, I thought classpath meant both path and deps #2018-09-13 18:37:51 richiardiandrea as customary in all the previous tools 😉 #2018-09-13 18:38:24 seancorfield You mostly want -R or -M but if you are bringing in test code you probably want -C for that 🙂 #2018-09-13 18:38:59 seancorfield What I recommend is not having :main-opts in the same alias as anything else -- use a separate alias that is just for running that main. #2018-09-13 18:39:30 seancorfield For example, in several projects I have :test to set everything up for testing and :runner to actually run the tests. #2018-09-13 18:40:10 richiardiandrea so I am using new and telling my team to just do clj -A:new:tmpl cljs-template org-scope/my-project #2018-09-13 18:40:14 seancorfield I tend to keep all my aliases small and orthogonal -- so they can be composed 🙂 #2018-09-13 18:40:31 richiardiandrea but now I am just in need of a REPL with the deps #2018-09-13 18:40:40 richiardiandrea so I do not want -M I guess 😉 #2018-09-13 18:40:56 seancorfield Which is exactly why you want the main opts separate from the deps 🙂 #2018-09-13 18:41:01 richiardiandrea Anyways, this works clojure -R:gen:new:tmpl -A:nrepl #2018-09-14 15:22:31 niclasnilsson Using deps.edn, when developing an app with some libs locally, what’s the intended way when App -> LibA and LibA -> LibB. Is there a way to do this without tainting App’s deps.edn with App’s dependencies dependencies (where all are local)? #2018-09-14 15:23:14 dominicm @niclasnilsson :local/root I think? #2018-09-14 15:23:21 mpenet you can have a profile with local dependencies #2018-09-14 15:23:26 mpenet I mean an alias #2018-09-14 15:23:47 niclasnilsson @dominicm Only seems to work in one “layer”, not picking things up recursively #2018-09-14 15:24:21 niclasnilsson @mpenet, yep, that’s what I’d like to avoid. It’s just boring. #2018-09-14 15:24:23 mpenet well you can add exclusions and explicit deps to other local deps #2018-09-14 15:24:39 mpenet but yeah, it requires some work #2018-09-14 15:24:55 dominicm @niclasnilsson I see, you mean when all 3 are local. Transitive aliases don't work, I don't think there's anyway to configure libraries to load in transitive local mode. #2018-09-14 15:25:55 niclasnilsson And (I think) even when LibA is local and LibB is on github for instance, App doesn’t seem to pick it up. (Unless I did something wrong) #2018-09-14 16:02:47 richiardiandrea The local dep issues are quickly becoming a problem for our Cursive users - we spend a lot of time clicking on the error dialogs popping up - we tried adding :manifest :deps but the the use from the command line broke for some reason.
I might invest some time digging into it... #2018-09-14 16:03:48 cfleming @richiardiandrea I think you want :deps/manifest :deps #2018-09-14 16:04:34 richiardiandrea Ah yep I think with put that sorry 😃 will come up with a repro first then we'll see #2018-09-14 17:15:45 alexmiller @niclasnilsson don’t forget that you can put aliases in ~/.clojure/deps.edn too and then combine them with stuff in your project. so if you had a bunch of local dep overrides, you could declare them there (without touching your project deps.edn) #2018-09-14 23:13:28 jcf Something that just occurred to me. Languages like Ruby and Perl that have spent a lot more time optimising their CLIs have relatively succinct (but also somewhat cryptic) APIs for interactic with text. Consider the following examples, first in Ruby and the an approximation using clj:
$ echo 4 | ruby -pne '$_.to_i + 1'
4
$ echo 4 | clj -e '(inc (Long/parseLong (first (line-seq (java.io.BufferedReader. *in*)))))'
5
I'd argue that $_ is hard to remember and Google, but if you want to apply an operation to each line in some input the Ruby syntax is quite convenient (assuming you can remember it).
Are there plans on adding similar switches to the new clj executable to support process *in* with a bit less ceremony? #2018-09-14 23:13:47 alexmiller there is one #2018-09-14 23:13:55 alexmiller just append - at the end #2018-09-14 23:15:31 jcf Hi @alexmiller! How would I change echo 4 | clj -e '(inc (Long/parseLong (first (line-seq (java.io.BufferedReader. *in*)))))' to make use of -? Sorry for being dumb but it's not immediately obvious to me. #2018-09-14 23:15:50 jcf I have used - with other programs before, but not with clj. #2018-09-14 23:17:58 jcf main-opts
-m, --main ns-name
Call the -main function from namespace w/args
-r, --repl
Run a repl
path Run a script from a file or resource located at path
- Run a script from standard input
#2018-09-14 23:18:35 jcf I've found - in the man page for clj. #2018-09-14 23:22:04 jcf For context I'm thinking I want to make it easier doing something like curl | jq '.users' | clj -e '...' so I can combine Clojure with other tools I have on the command-line. #2018-09-14 23:22:52 jcf I can of course do everything from Clojure, but just wondering if there's thought going into the API for passing in data to a new Clojure process. #2018-09-15 00:09:57 seancorfield @jcf Is this still too much code? (! 550)-> echo 4 | clj -e '(-> (read-line) (Long/parseLong) inc)'
5
#2018-09-15 00:13:47 seancorfield If you want to read every line: (! 552)-> (echo 4; echo 13) | clj -e '(while (some-> (read-line) (Long/parseLong) (inc) (doto (println))))'
5
14
#2018-09-15 00:25:31 jcf Definitely more terse than my snippet. I like the use of read-line and a threading macro for sure!
I think we're missing a trick still by not having the switches provided by Ruby.
The whole loop-and-read boilerplate becomes redundant when you pass -n, and with -p you don't have to print…
#2018-09-15 00:31:05 seancorfield Clojure is a fundamentally different execution model tho': you're building a classpath, firing up a JVM, compiling a set of forms, and then running the compiled code. #2018-09-15 00:31:54 seancorfield With Ruby, it can just take the expression and interpret it repeatedly... #2018-09-15 00:33:02 seancorfield ...for Clojure, you'd need some in-JVM process that takes your expression and does the looping/reading for you -- you can't do it at the script level. #2018-09-15 01:10:42 seancorfield OK, @jcf how about this for Ruby-like conciseness? (! 584)-> (echo 4; echo 13) | clj -Apne -e '($ (inc (Long/parseLong $_)))'
#'user/$
5
14
#2018-09-15 01:12:41 seancorfield $ is a macro defined by the pne alias that takes the following expression and evaluates and prints it repeatedly in the context of $_ bound to subsequent (read-line) calls... #2018-09-15 01:13:35 seancorfield As for the pne alias... be afraid... be very afraid... (! 585)-> cat deps.edn
{:aliases,{:pne,{:main-opts,["-e","(defmacro,$,[&,body],(list,'while,(list,'when-let,'[$_,(read-line)],(let,[v,(gensym)],(list,'let,(vector,v,(list*,'do,body)),(list,'println,v))),'$_)))"]}}}
#2018-09-15 01:14:35 seancorfield (you can't use backtick easily with clj due to bash issues so you have to laboriously build the entire body with list and vector -- and you have to use , for whitespace in :main-opts because of reasons... 🙂 ) #2018-09-15 07:26:12 jcf That is gnarly, but I love the novelty of your solution.
I'm guessing a solution with similar behaviour to this won't land in tools.deps, but you've inspirited me to think about writing a small library that adds this functionality.
Thanks, @seancorfield! #2018-09-15 06:10:17 dottedmag @seancorfield Maybe instead of that one can add a dependency that populates user and make :main-opts require it? #2018-09-15 06:13:48 seancorfield @dottedmag I'm sure there are several ways to solve the problem... this was just the first thing that popped into my head 🙂 #2018-09-15 06:19:18 seancorfield Feel free to post your solution too... #2018-09-16 23:27:39 seancorfield @bhauman Thanks for adding that deps.edn -- see my comments in the now-closed issue... also my comment on the most recent commit (which introduces a syntax error). #2018-09-16 23:33:30 seancorfield Is it a known bug that a local dep inside a git dep doesn't work properly? I expected tools.deps to find the deps.edn inside the cloned repo via the local/root... but it doesn't seem to 😞 #2018-09-16 23:34:03 alexmiller Yes #2018-09-16 23:40:07 seancorfield I couldn't find the JIRA ticket for that @alexmiller -- I assume there is one? #2018-09-16 23:51:27 alexmiller There are several I think - it’s all the local dep path stuff #2018-09-17 00:12:44 seancorfield TDEPS-74 seems to get to the heart of it. OK, I guess I'll go vote for it! #2018-09-17 23:46:12 seancorfield @alexmiller @arrdem was offering a patch for the local/root issue I believe... #2018-09-17 23:46:52 alexmiller well I’ve thought about it a lot and haven’t come up with a solution I’m happy with #2018-09-17 23:47:18 seancorfield If the tools.deps reader converted paths (from :paths, :extra-paths, and :local/root -- and maybe others?) from relative to absolute, based on the location of the deps.edn file in which they occur, would that solve the problem? #2018-09-17 23:47:34 alexmiller I don’t think so #2018-09-17 23:47:47 seancorfield (or, rephrased, at what point does tools.deps convert paths to absolute ones -- they are absolute in the classpath at the end) #2018-09-17 23:48:26 alexmiller the extension does it when it resolves the paths for a dep #2018-09-17 23:49:28 alexmiller but it’s really about what the “current directory” means when you are resolving a transitive dep by relative path - currently nothing is shifting that context so you always have the root dep context #2018-09-17 23:52:45 seancorfield Ah, and paths can be in both the system deps and the per-user deps and of course you don't want those resolved against the file they're defined in... yeah... #2018-09-17 23:53:18 seancorfield And by the time you've collapsed all the deps down, you've lost that context, right? #2018-09-17 23:56:22 alexmiller yeah. we actually conceived local deps primarily as working primarily with absolute paths - all the stuff people are doing with relative paths and monorepos is somewhat emergent #2018-09-17 23:59:25 arrdem So for https://github.com/fundingcircle/lein-modules, we added a “//” notation somewhat stolen from pantsbuild - the idea was that paths which were not “//” prefixed were treated as relative and just passed around, “//” prefixed paths were treated as being absolute with respect to the project.clj the path occurred in - for us implicitly the “root” project.clj of the repo which emulates the meaning of “//” in Google Blaze derived systems where “//” is repo root when referring to build target names. #2018-09-18 02:56:14 alexmiller Yeah, I’m not doing that :) #2018-09-18 05:21:07 arrdem 😉 #2018-09-18 05:35:17 arrdem Perhaps a more serious option here would be #file EDN extensions... maybe not suitable for general purpose but perhaps useful for us. #2018-09-18 05:42:41 alexmiller None of that seems necessary to me #2018-09-18 05:43:15 alexmiller I think the intent of this kind of setup is clear as is, the questions are in the implementation #2018-09-17 23:59:36 arrdem That’s worked out OK at best for us. #2018-09-18 00:05:23 arrdem file me under monorepo user probably pushing the boundaries of expected behavior. #2018-09-18 00:05:48 arrdem I’m currently exploring replacing lein-modules for our mid-size monorepo for a couple reasons. #2018-09-18 01:14:33 richiardiandrea couldn't we have a placeholder deps.edn in the root of a monorepo and subdeps consider that first if present for :local/root? #2018-09-18 04:55:59 dominicm I did wonder if relative paths would be a different key. #2018-09-18 17:54:29 arrdem Chatted about this with a coworker some and I think we’re gonna take a swing at adding a resolver type which patches up paths, everything else seeming pretty janky in comparison. We’ll see how that goes. #2018-09-18 18:07:37 alexmiller what do you mean by a resolver type? #2018-09-18 18:14:59 arrdem Extension / coordinate type. #2018-09-18 19:06:27 alexmiller I feel like you’re going down a path that is a waste of time #2018-09-18 19:07:09 alexmiller Which you’re welcome to do, but there is no support for installing your own extensions in clj, so you’ll only be able to use this with some frankensetup #2018-09-18 19:11:06 dominicm Will there ever be support for custom extensions? #2018-09-18 19:30:43 alexmiller undecided #2018-09-18 19:31:09 alexmiller I mean, you can use them now if you use tools.deps.alpha as a library #2018-09-18 19:31:55 alexmiller it’s just yet another layer of installation complexity when it comes to clj #2018-09-18 20:20:10 dominicm I suppose it would require t.d.a to orchestrate itself in order to run again with additional dependencies. Probably not completely horrible if the context allows add-lib. #2018-09-22 14:40:55 raymcdermott @alexmiller just checking that the add-lib method from earlier SHAs should still work with 1.10-alpha7 #2018-09-22 14:41:05 raymcdermott cos I am having fails #2018-09-22 14:44:49 alexmiller Haven’t tried it, but not sure why it wouldn’t off the top of my head #2018-09-22 14:44:54 alexmiller Fails how? #2018-09-22 14:45:07 raymcdermott gives back a false #2018-09-22 14:45:18 raymcdermott for the example memoize (and others) #2018-09-22 14:46:04 raymcdermott could be something on my end - just wanted to make sure that it wasn’t meant to be deprecated #2018-09-22 14:49:00 alexmiller $ clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-alpha7"} org.clojure/tools.deps.alpha {:git/url "" :sha "d492e97259c013ba401c5238842cd3445839d020"}}}'
Downloading: org/clojure/clojure/1.10.0-alpha7/clojure-1.10.0-alpha7.pom from
Downloading: org/clojure/clojure/1.10.0-alpha7/clojure-1.10.0-alpha7.jar from
Clojure 1.10.0-alpha7
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'org.clojure/core.memoize {:mvn/version "0.7.1"})
true
user=> (require 'clojure.core.memoize)
nil #2018-09-22 14:50:07 raymcdermott maybe false is OK if the lib is already available #2018-09-22 14:50:38 alexmiller yeah, you’ll get true only if added a new lib(s) #2018-09-22 14:52:34 raymcdermott ok, great - thanks for checking #2018-09-22 14:55:55 alexmiller it can’t be deprecated - I haven’t even released it yet :) #2018-09-22 14:58:10 raymcdermott LOL fair point #2018-09-22 14:58:29 raymcdermott are you close to release or is the debate still ‘raging’? #2018-09-22 15:00:10 alexmiller been focused on other things :) #2018-09-22 15:02:31 raymcdermott what could possibly be more important #2018-09-22 15:58:16 raymcdermott the problem I’m having might be related to Cursive as the tests all pass via the CLI #2018-09-22 15:58:53 raymcdermott and fail via the deps.edn default REPL there, so I’ll mention it on that channel later #2018-09-24 08:35:51 martinklepsch i.e. the only jar for the version has a -javadoc classifier #2018-09-24 12:06:29 alexmiller What are you expecting to happen? #2018-09-24 12:06:58 alexmiller What is your deps.edn? #2018-09-30 08:07:43 alan Hi, sorry for the late response, I raised the issue because while I can build without any issue with leiningen I couldn't make the cljdoc build (which uses tools.deps) work #2018-09-30 08:11:30 alan This is the tools.deps {:deps {org.clojure/clojure #:mvn{:version "1.9.0"}, ml.dmlc/xgboost4j-example #:mvn{:version "0.72"}, ml.dmlc/xgboost-jvm #:mvn{:version "0.72"}, org.clojure/clojurescript #:mvn{:version "1.10.339"}, org.clojure/tools.namespace #:mvn{:version "0.2.11"}, clj-boost/clj-boost #:mvn{:version "0.0.1"}, org.clojure/java.classpath #:mvn{:version "0.2.2"}, org.clojure/data.csv #:mvn{:version "0.1.4"}, ml.dmlc/xgboost4j #:mvn{:version "0.72"}, codox/codox {:exclusions [enlive hiccup org.pegdown/pegdown], :git/url "", :sha "4b0720941083fda9643d905f0854fabea55b175f", :deps/root "codox/"}}} #2018-09-30 08:13:00 alan It seems impossible to get the XGBoost stuff in this way. While in leiningen I have [ml.dmlc/xgboost4j "0.72" :extension "pom"] and everything's fine #2018-09-30 12:51:33 alexmiller pom extension is not supported - what are you expecting that to do? #2018-09-30 13:18:14 alan I'm not using it, is in cljdoc to build the package, but now I guess there's not much it can be done about it #2018-09-30 14:03:12 alexmiller Sorry, I’m not understanding the problem. It would be very helpful to have a description of the problem, with the things needed to reproduce it, what you expect to happen, and what is actually happening. I feel like the above has half of that but it’s very confusing to me what you’re doing. #2018-09-30 14:07:33 alan I have a dependency, the only way to get it is to use the :extension "pom" flag, I develop, build and release my library with lein, no issues. I wanted to build docs with cljdoc which uses tools.deps and the build fails because of that dependency #2018-09-30 14:21:48 alexmiller “the only way to get it is to use the :extension "pom" flag” - why? #2018-09-30 14:25:54 martinklepsch @alexmiller because the dependency doesn’t come with a regular .jar file — I don’t know too much about why you package libraries like this but it seems to be a thing in this case
And as far as I understand nobody (using tools.deps) will be able to use @U7P0DPQ1H s library which depends on this artifact #2018-09-30 14:37:49 alexmiller If it doesn’t have a jar, then i don’t get how it affects the classpath. #2018-09-30 14:39:42 alexmiller I guess maybe by having deps #2018-09-30 14:40:32 alexmiller Which I would actually expect to work #2018-09-30 14:41:03 alexmiller If someone could file a jira in TDEPS about this, I will look at it more closely later this week #2018-09-30 14:56:53 martinklepsch Yeah would assume that the point of this kind of artifact is usually its deps, I.e. it’s a kind of Uber-module #2018-09-30 14:57:59 martinklepsch wont have time to file a ticket in the next few days... #2018-09-24 12:04:20 alexmiller Are you using the latest release? #2018-09-24 12:05:28 alexmiller And what is your deps.edn? There is some new syntax from TDEPS-12 with an example in the comments there. #2018-09-24 12:09:20 alexmiller Specifically you’ll want the lib just once with a coordinate that has :classifier [“” “native”] #2018-09-24 12:13:23 alexmiller {:deps {com.github.jnr/ffi {:mvn/version “1.2.16” :classifier [“” “native”]}}} #2018-09-24 12:28:56 dominicm @bherrmann that surprises me for two reasons, one is that it should be "src", the other is that it's a default. #2018-09-24 12:29:04 dominicm Where did you put hello.clj? #2018-09-24 12:35:00 bherrmann ok, yea. hello.clj just needs to be under "src" #2018-09-24 12:35:22 bherrmann my bad... I'm automating a manual process (converting and excel file) and was thinking more script minded, less software development minded (where a src sub directory is more assumed) #2018-09-24 14:06:09 alexmiller @mkvlr I looked at this and have reopened - you’re right that this is related to TDEPS-12 but needs a bit more work #2018-09-24 14:06:17 alexmiller but I’m not going to get to that this week #2018-09-24 14:09:06 mkvlr @alexmiller thanks a lot! Yes, we can work around by manually specifying it for now so no rush from us. It’s just missing to pick up the classifier when reading the pom if I understand it correctly. #2018-09-24 14:13:02 bhauman I’m writing some docs on resolving dependency conflicts, is there a tools deps equivalent to lein pedantic? #2018-09-24 14:15:19 alexmiller yes #2018-09-24 14:33:35 bhauman thanks @alexmiller, while that works and is good to know about it implements lein-ancient it doesn’t find dependency conflicts like pedantic 🙂 #2018-09-24 14:33:49 alexmiller oh, sorry. then no :) #2018-09-24 14:34:24 bhauman -Stree helps of course #2018-09-24 14:34:42 alexmiller there is a totally unsupported subject to removal and alteration verbose mode for tools.deps that is primarily designed to be helpful to me #2018-09-24 14:35:15 bhauman oh I’ll put that in the docs then 😉 #2018-09-24 14:35:38 alexmiller definitely an area where more could be done #2018-09-24 14:36:26 bhauman I moved figwheel to Jetty thinking it was a smart move to get https support and to play nicely with larger environment #2018-09-24 14:36:42 bhauman but dang its prone to tricky conflicts #2018-09-24 14:37:13 bhauman the way the package is divided up #2018-09-24 14:37:29 bhauman when I say tricky I mean tricky for newcomers #2018-09-24 14:38:23 bhauman the conflict between ring and datomic cloud is the main problem #2018-09-24 16:22:37 lilactown yeah I spent about 3 hrs early on trying to get Jetty (for local dev) and datomic cloud working. I switched to http-kit and it is working well so far #2018-09-25 21:55:49 bhauman @U4YGF4NGM I’ve got docs on this now #2018-09-25 21:56:46 bhauman published it this weekend #2018-09-25 21:56:52 bhauman sorry about that #2018-09-25 21:57:03 lilactown awesome! ty! #2018-09-24 21:05:13 seancorfield If I have some.group/artifact {:mvn/version nil} and an :override-deps entry of some.group/artifact {:mvn/version "1.2.3" :exclusions [foo/bar com.quux/blah]} should I expect the :exclusions from the override to be applied to the main dep? #2018-09-24 21:05:32 seancorfield (and what happens if there are :exclusions on both the main dep and the override?) #2018-09-24 21:05:41 alexmiller the override overrides #2018-09-24 21:06:26 alexmiller as in replace the original (including its exclusions) #2018-09-24 21:06:47 seancorfield Cool. So I can rely on specifying :exclusions just once in the overrides -- that's excellent! #2018-09-27 07:07:34 tianshu when try to uberjar, meet this error
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.quartz-scheduler:quartz-parent:pom:2.3.0 in terracotta-snapshots ()
This dependency, will be handled correct when using leiningen. I don't know how to investigate the issue, where should I start? #2018-09-27 12:21:39 dominicm This looks like TDEPS-50 also? #2018-09-27 13:44:35 orestis Is there a tools.deps friendly way to run Eastwood from the command line? #2018-09-27 14:35:43 orestis Also, what is a no-fuss way to build an uberjar for a deps.edn project? I tried pack.alpha, depstar so far and I get weird errors. #2018-09-27 14:37:12 orestis pack.alpha: NOTE: /alpha/bootstrap/onejar/src/com/simontuffs/onejar/JarClassLoader.java, line -1: Some input files use unchecked or unsafe operations.
NOTE: /alpha/bootstrap/onejar/src/com/simontuffs/onejar/JarClassLoader.java, line -1: Recompile with -Xlint:unchecked for details.
~/d/n/c/october (master|✚1…) $ java -jar output.jar
Exception in thread "main" java.lang.ClassNotFoundException: clojure.main
at com.simontuffs.onejar.JarClassLoader.findClass(JarClassLoader.java:713)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at com.simontuffs.onejar.JarClassLoader.loadClass(JarClassLoader.java:630)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.simontuffs.onejar.Boot.run(Boot.java:336)
at com.simontuffs.onejar.Boot.main(Boot.java:168)
#2018-09-27 16:37:11 seancorfield @orestis For pack.alpha, that seems to indicate it didn't include the Clojure runtime -- no idea about that since I haven't tried it but @U09LZR36F (the author) is very helpful. For depstar, adding the manifest manually is an easy step and I like the raw simplicity of it. Haven't tried cambada so thanks for finding that and posting the link! #2018-09-27 16:38:53 orestis It’s up on the tools.deps wiki and has a very good README :) #2018-09-27 16:39:27 seancorfield Yeah, just looking it over. Very nice! #2018-09-27 16:59:31 dominicm I think the pack bug would be fixed by using the latest version. #2018-09-27 16:59:41 dominicm I haven't updated the installer, and I should! #2018-09-27 16:32:20 andy.fingerhut I do not have a tools.deps friendly way handy and worked out for running Eastwood from the command line, but if you look at this section of the Eastwood docs on how you can do it from the REPL, without Leiningen involved, you or someone better at tools.deps knowledge than I might be able to figure it out. I'd be happy to add a note like that to the Eastwood README if it isn't too difficult to explain: https://github.com/jonase/eastwood#running-eastwood-in-a-repl #2018-09-27 16:33:58 andy.fingerhut Warning: The latest Eastwood 0.2.9 release has a bug in which when you try to use it, it will download (if you don't already have it) and use Eastwood 0.2.8 instead. You may want to simply specify Eastwood 0.2.8 for now, since that is what you will end up using anyway. #2018-09-27 16:47:44 seancorfield @orestis clj -Sdeps '{:deps {jonase/eastwood {:mvn/version "0.2.8"}}}' -e "(require '[eastwood.lint :as e])" -e '(e/eastwood {:source-paths ["src"] :test-paths ["test"]})' #2018-09-27 16:48:14 seancorfield You could squirrel most of that away in your ~/.clojure/deps.edn file as an alias... #2018-09-27 16:51:59 seancorfield :eastwood {:extra-deps {jonase/eastwood {:mvn/version "0.2.8"}}
:main-opts ["-e" "(require,'[eastwood.lint,:as,e])"
"-e" "(e/eastwood,{:source-paths,[\"src\"],:test-paths,[\"test\"]})"]}
(note that you need , instead of space!) #2018-09-27 18:09:41 orestis I ended up doing this: :eastwood {:extra-deps {jonase/eastwood {:mvn/version "0.2.8"}}
:main-opts ["-m" "eastwood.runner"]
:extra-paths ["test" "tools"]}
And I made a tools directory to contain simple namespaces like that. In this case: (ns eastwood.runner
(:require [eastwood.lint :as e]))
(defn -main [& args]
(e/eastwood {:source-paths ["src"] :test-paths ["test"]})) #2018-09-27 18:55:16 seancorfield Oh, that's nice! 👍:skin-tone-2: #2018-09-27 17:01:46 orestis Yeah, I saw that trick on your dot-Clojure and was about to ask! Is that a known bug? #2018-09-27 17:02:07 orestis I guess though I can just as easily make a tiny namespace to do this. #2018-09-27 17:07:19 seancorfield @orestis It's to do with how the strings are run through the clojure shell script and into the main opts cache file and then read back. So, yeah, it's a "known feature" 🙂 #2018-09-27 17:07:54 orestis Right, so it’s not going away any time soon I guess? #2018-09-27 17:08:50 orestis I remember some mentions of caching compiled dependencies to speed up startup time - any ideas about that? #2018-09-27 17:10:45 orestis More questions - in deps.edn, can you actually override the entire top-level deps entry? Or can you only append? #2018-09-27 17:12:45 orestis Rationale: most of my aliases so far have to do with running lint tools, packaging, test runners etc. These namespaces have no need to access my actual app’s classpath. I guess there has bound to be some overhead when the class path is unnecessarily large? #2018-09-27 17:13:08 orestis I know that clj will cache the classpath and it’s actually very noticeable. #2018-09-29 04:07:41 miridius Hi all, what's the best practice for running a deps.edn based project in production? Is it ok to run it "dynamically" with clojure -Aprod -m my.app.main or is it better to AOT compile the app -- and if so, how? #2018-09-29 06:06:05 seancorfield AOT really only saves you startup time. #2018-09-29 06:06:24 dominicm We do that sometimes. No complaints. #2018-09-29 06:06:28 seancorfield We still use JAR files for production deployment but we never AOT anything. #2018-09-29 06:07:25 seancorfield We use uberjars for the convenience of knowing our production system will never be dependent on reaching out to Maven or Clojars at application startup. But we use clojure.main as our Main-Class. #2018-09-29 06:10:34 miridius what do you use for creating the uberjar? juxt/pack.alpha -> Capsule? #2018-09-29 06:10:58 seancorfield We're still using Boot but we're moving toward clj/`tools.deps`. #2018-09-29 06:11:30 seancorfield I'm leaning toward depstar for the simplicity of the tooling, but cambada seems to be very slick. #2018-09-29 06:11:41 miridius ah ok 🙂 thanks. I'm currently using lein for the uberjar task and literally nothing else #2018-09-29 06:13:01 seancorfield We outgrew Leiningen years ago and switched to Boot -- we've built almost our entire build/test/deploy pipeline on top of that. #2018-09-29 06:13:20 seancorfield But we're in the process of retooling to something simpler, based on deps.edn etc. #2018-09-29 06:13:29 miridius depstar and cambada both look really cool! thanks for those #2018-09-29 06:14:57 miridius Yeah I've tried both leiningen and boot and found them both to increase the amount of complexity I face rather than decrease it. Troubleshooting them when they don't behave is especially painful! So I've happily moved everything to tools.deps, and the only piece missing now is production deployment #2018-09-29 06:15:33 seancorfield Our build.boot file is close to 2,000 lines. We have a ways to go 🙂 #2018-09-29 06:15:39 miridius wow... #2018-09-29 06:17:15 miridius I use docker so theoretically don't have any need to uberjar anything, as long as I can make sure all the deps are there. I was thinking of adding a :quit command-line arg to my main class that quits immediately and just running that at build time so that it downloads all the deps. Kind of hacky though... #2018-09-29 06:18:17 seancorfield Can't you do clj -Stree to force the downloads? #2018-09-29 06:20:34 miridius oh! I didn't know it would download the deps, I thought it just prints them #2018-09-29 06:20:42 miridius hey btw, I use your dot-clojure repo, it's really helpful 🙂 #2018-09-29 06:21:51 miridius -Stree works... nice #2018-09-29 08:15:28 dominicm We (JUXT) use pack in many projects now, it's gone very well overall. #2018-09-29 06:07:45 seancorfield ^ @miridius #2018-09-29 06:09:21 miridius thanks for the tips 🙂 #2018-09-29 21:15:04 bbrinck Hi, I just made a small utility (using clj-new- thx @seancorfield) to validate deps EDN from stdin - https://github.com/bhb/check-deps. Feedback/questions welcome 🙂 #2018-09-30 02:12:46 richiardiandrea Nice! #2018-09-29 21:17:09 bbrinck (and suggestions) #2018-09-30 12:47:08 krukow Running into an issue migrating from lein to tools deps. REPL works, but when I try to uberjar the project I get the following error:
Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
https://gist.github.com/krukow/b2b6ef6536cb8cf07cc8f613afaa8108
Would someone here be kind enough to help be figure out what's going on? #2018-09-30 12:51:28 krukow Hmmm - resolved by cleaning .m2 and adding
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2018-09-30 12:51:55 krukow did this mean I had a corrupt .m2 #2018-09-30 12:55:03 alexmiller Those repos are already included by default so shouldn’t need those #2018-09-30 12:55:14 alexmiller But maybe a bad m2 #2018-09-30 12:58:31 krukow thanks Alex - I'll remove and simplify, I think I'll clean the entire .m2 and re-build #2018-09-30 12:58:56 krukow migration from lein to deps was very easy 🙂 thanks! #2018-09-30 13:00:42 krukow wait - this is weird ... I'll double check but what I'm seeing now is #2018-09-30 13:02:23 krukow removing :mvn/repos from deps.edn => results in the above ArtifactDescriptorException: exception #2018-09-30 13:03:38 krukow adding
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
=> uberjar builds #2018-09-30 13:07:06 krukow not happy about using http though #2018-09-30 13:09:52 krukow Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:323)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:192)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:253)
at clojure.tools.deps.alpha.extensions.maven$eval2646$fn__2648.invoke(maven.clj:79)
at clojure.lang.MultiFn.invoke(MultiFn.java:243)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:170)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:152)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:215)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
at cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109)
#2018-09-30 13:11:01 krukow this is independent of the state of the .m2 dir #2018-09-30 13:12:16 krukow seems that this call cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109) => clojure.tools.deps.alpha$resolve_deps behaves differently depending on whether I have
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
or not in deps.edn #2018-09-30 13:12:19 krukow strange #2018-09-30 20:25:40 seancorfield @krukow I just tried with your deps.edn and that build command and it worked just fine. I got this warning tho' Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method, or the namespace has not been AOT-compiled.
#2018-10-01 14:14:21 krukow @seancorfield thanks for helping out - I ended up with this uberjar command:
:uberjar
{:extra-deps
{luchiniatwork/cambada {:mvn/version "1.0.0"}}
:main-opts ["-m" "cambada.uberjar"
"-m" "vsts-flow-metrics.cli"]}}
which should help for the main warning.
Then without specifying a :mvn/repos in deps.edn I get:
Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:323)
...
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:197)
at cambada.uberjar$get_dep_jars.invokeStatic(uberjar.clj:109)
But If deps.edn contain
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
the command completes #2018-10-01 14:14:31 krukow I'll see if I can create a minimal repro #2018-10-01 14:14:53 krukow and I use clj -A:uberjar -Srepro -Sforce to create the uberjar #2018-10-01 14:46:42 timgilbert Say, using an s3p private repo I get a lot of messages like these when starting clojure:
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Apparently this file is just missing, but I definitely don't have it in my private repo. Is there any way to tell clojure and/or maven to only use the private repo for artifacts from a specific namespace? #2018-10-01 16:05:32 timgilbert Alternately, is there a way to suppress that message from appearing? It's not so good for shell-script type tasks #2018-10-01 16:08:19 timgilbert The full panoply of messages is more like this:
Downloading: com/google/code/findbugs/jsr305/maven-metadata.xml from
Downloading: com/google/guava/guava/maven-metadata.xml from
Downloading: commons-codec/commons-codec/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-kinesis/maven-metadata.xml from
Downloading: com/amazonaws/amazon-kinesis-client/maven-metadata.xml from
#2018-10-01 19:08:48 zane :jvm-opts isn't a valid top-level key, right? Only inside an alias in :aliases? #2018-10-01 19:43:04 alexmiller correct #2018-10-01 19:43:23 alexmiller there is a ticket wrt default jvm-opts #2018-10-01 20:31:28 zane Okay. Thanks, Alex! #2018-10-02 05:45:42 kommen is there a way to figure out in a clojure process started with clj which aliases were used to start it? #2018-10-02 06:10:10 seancorfield @kommen No. #2018-10-02 06:11:00 seancorfield The aliases are only used to build the classpath (and main opts etc). By the time your program is running, none of that is available. #2018-10-02 06:11:16 seancorfield What is the problem you're trying to solve here? #2018-10-02 06:13:43 kommen cider checks if dependencies for certain functionality are on the classpath. until now, simply by splitting the classpath entries and interpreting that. this falls apart when using tools.deps git and local coordinates #2018-10-02 06:15:30 seancorfield Yup. Well, I think CIDER is trying to be far too clever here and making all sorts of unwarranted assumptions -- but it sounds like Bozihdar is aware of the problem and will figure it out -- he's been doing amazing stuff with CIDER... #2018-10-02 06:16:47 seancorfield clj and tools.deps are making a lot of people reconsider their assumptions 🙂 #2018-10-02 06:23:24 kommen ok, thanks #2018-10-02 06:49:58 dominicm The safe way to do it is to check for the existence of namespaces/classes, not to search the classpath for them. #2018-10-02 20:13:31 krukow @seancorfield @alexmiller I have a simple repro (on my machine) - from a clean install. Unzip the .zip file into an empty directory and run clj -A:uberjar -Srepro -Sforce - would any of you be so kind as to confirm / invalidate the repro so I can make a formal issue? #2018-10-02 20:15:58 krukow FYI the repro sequence was: with an empty /.m2, /.clojure on OS X high sierra with brew install clojure, `clj -Sdeps '{:deps
{seancorfield/clj-new
{:git/url "https://github.com/seancorfield/clj-new"
:sha "21ca1b27f46dc324be084ba839beca555aeda387"}}}' \
-m clj-new.create \
app \
krukow/artifact-exception` and then clj -A:uberjar -Srepro -Sforce in the folder where simple-repro.zip is unzipped. #2018-10-02 20:18:30 krukow then add
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2018-10-02 20:18:37 krukow to deps.edn and things work #2018-10-02 20:34:52 dominicm @krukow this makes sense to me. Cambada doesn't consider the system deps.edn file. #2018-10-02 20:36:12 seancorfield Ouch! That's quite the gotcha! #2018-10-02 20:43:36 krukow thank you! #2018-10-02 20:44:31 krukow but the workaround is easy then - I should have thought of this 😮 #2018-10-02 20:44:55 krukow One more general question: where can I find the source code for the clojure & clj CLI tools #2018-10-02 20:46:04 seancorfield The brew-install and tools.deps.alpha repos under the clojure org. #2018-10-02 20:46:35 seancorfield (the former is the shell scripts etc, the latter is the library they use for all the dependency stuff) #2018-10-02 20:47:33 krukow 👍 thank you Sean #2018-10-03 05:09:22 dominicm Also means it doesn't need to merge in paths #2018-10-03 05:29:51 zane Hoping to get one more set of eyes on this before I file an issue. #2018-10-03 05:46:21 seancorfield It's missing var as the second argument to the map call. Def. a bug. #2018-10-03 13:51:21 zane That was my guess too. Thanks! #2018-10-03 07:06:43 krukow 👍 amar! #2018-10-03 14:49:26 hlolli I added this into my deps.edn :jvm-opts ["-Xms2G" "-Xmx5G"] started repl with cider, but I can't in any way see if these options are present. This is not present in the command line args if I look at the java process in my system monitor. #2018-10-03 15:06:24 ghadi I don't think these take effect except within an alias #2018-10-03 15:13:21 hlolli oh ok, good to know 🙂 @ghadi #2018-10-03 15:26:13 orestis @hlolli see in #cider for a way to start cider with a specific alias. #2018-10-03 16:03:35 hlolli bingo works, I've been bit turned off by dir-locals due to "unsafe" popup. But the popup didn't appear this time. #2018-10-04 06:36:50 dominicm I think I understand why, but should it be the case that, if an alias specifies main-opts (mine has -m in), then -e and -i are ignored? The output of clj --help doesn't exactly suggest that -e and -m are mutually exclusive, especially as they are "different".
Currently cider tries to use -e in order to start nrepl without interrupting the users choice of main-opts. #2018-10-04 06:39:21 dominicm One solution might be to add a java class to cider releases which triggers in response to a jvm option, but that's not going to be my first approach if I can help it. -e is just lovely as a general tool. #2018-10-04 12:36:23 genekim Not sure where I should ask this, but wanted to put this out there —
After listening to a @apropos_cast, I took the plunge and tried clj and tools.deps.
I was blown away by how easy it was! I used lein-tools-deps by @rickmoynihan — it took only 10m to transition my first project over, and was able to quickly do three other lein projects. It was fantastic to be able to share library code w/o publishing to local maven repo. It made using the monorepo pattern fantastically simple.
Marvelous work, Rick — and tools.deps is fantastic!
However, on my laptop, I found that “clj -m foo.main” takes 15s to startup, 2x longer than “lein run”. (Which is not what I expected. I expected startup times to be the same, or lein to be worse, given my understanding that there’s one more JVM being started?)
It’s not in a public repo, so it’d take some time to come up with something that you could reproduce — but if anyone is interested, I can capture some traces, or with some help, I can try to come up with a reproducible test case? #2018-10-04 13:06:34 rickmoynihan Alex Miller deserves almost all of the credit. ltd does almost nothing. But thanks for the thanks. #2018-10-04 13:52:06 alexmiller On the timing, was that first time or second time (after classpath is cached)? #2018-10-04 14:39:39 genekim @U064X3EF3 brilliant work! It’s a marvel! Thank you!
On timing: I ran all the timings 5-10 times, so I’m presuming this is with classpath caching. #2018-10-04 12:42:00 mpenet I think lein has default jvm flags optimised for startup time, clj I don't think does anything like that #2018-10-04 12:42:10 mpenet I could be wrong tho #2018-10-04 13:52:47 alexmiller I doubt you’ll see that much of a difference from those flags #2018-10-04 13:53:29 alexmiller If you’re using :aot with lein then that could be a big difference #2018-10-04 13:53:47 mpenet Yeah in general when people complain about startup time I reply that their workflow is prolly not really optimum #2018-10-04 13:53:48 alexmiller Lein would use the compiled classes #2018-10-04 13:54:26 mpenet doing cli tools with clojure is another matter, solved with other tools probably #2018-10-04 16:22:25 cfleming I’m having a problem using tools.deps as a library. If I don’t set the CWD of the process using it to the correct directory, then the manifest type detection is broken. #2018-10-04 16:25:28 cfleming Is that expected, a known bug, or something I should file an issue for? I can’t see anything immediately obvious in the tracker. #2018-10-04 16:36:32 alexmiller If you are using absolute paths, then I presume things are fine. If you are using relative paths, then it seems reasonable to expect that they are relative to the current working directory #2018-10-04 16:37:06 alexmiller Transitive local deps with relative paths are a problem and that’s a known issue #2018-10-04 16:37:53 cfleming I would have assumed that they would be relative to the file in which the ref was found. #2018-10-04 16:39:24 cfleming In this case, I have a root project which has a :local/root of “modules/something”. That deps.edn has another local ref with a relative path, which sounds like it’s a known issue? #2018-10-04 16:39:31 alexmiller That’s the known problem - TDEPS-74 #2018-10-04 16:40:46 cfleming I don’t think that’s the problem that I’m seeing, I’m seeing issues with manifest detection there too. #2018-10-04 16:41:00 cfleming But since the manifest detection is relative to the CWD, that will never work. #2018-10-04 16:43:06 cfleming I assume it’s all related to the same problem. #2018-10-04 16:48:24 cfleming I can’t see a good fix for this. detect-manifest would require the full path to the original file, and the only way I can see to pass that would be to add a new key to local coords when calling manifest-type. #2018-10-04 16:49:50 cfleming Even expand-deps doesn’t have that info right now, although I’m assuming that could be passed as a param. #2018-10-04 16:50:15 cfleming Ugh, in fact, not even resolve-deps has that info, since it’s passed a deps map, not the original files. #2018-10-04 16:53:10 alexmiller Yes, it’s difficult to fix, which is why it’s still open #2018-10-04 16:53:44 alexmiller I have a couple ideas on it to look at though #2018-10-04 16:53:59 cfleming It seems like the best place to fix it would be to absolutise those paths in read-deps. #2018-10-04 16:57:30 cfleming In fact, if that happened as part of canonicalisation in slurp-deps, then that should also work for transitive deps. #2018-10-04 16:59:38 alexmiller The related problem is that for local deps, it can’t build the proper set of deps.edn to merge - they’ve already been merged upstream #2018-10-04 17:00:12 alexmiller This relates to the notion of naming and separating the deps files #2018-10-04 17:00:14 cfleming The idea would be to merge local deps with the system and user deps files too? #2018-10-04 17:00:43 alexmiller Well the key is to NOT get the original project deps.edn #2018-10-04 17:00:57 cfleming Right. #2018-10-04 17:01:30 alexmiller And all of this stuff also is related to -Sdescribe and how things are installed on the various platforms #2018-10-04 17:02:01 cfleming The only way to reliably not get that is to pass either the paths or the read maps around, I guess, since merging is destructive. #2018-10-04 17:02:46 alexmiller I need some focus time to sit down and wrk through all these things. #2018-10-04 17:02:46 cfleming I guess you could merge system and user, and then pass that around and merge it with the other deps files, including the main project one, when you read them. #2018-10-04 17:03:28 cfleming That changes the whole API though, unless you stored that in the map you pass around I guess. #2018-10-04 17:03:55 cfleming Under a ::base-config key or something. #2018-10-05 10:01:54 mpenet I guess classpath-overrides is a possibility but that's a bit gross #2018-10-05 15:22:17 genekim Got it — thanks for the insights on lein compiles vs. clj, @alexmiller @mpenet ! #2018-10-05 17:38:09 adamfrey If I'm making a library with tools.deps and it includes a Java file in it, how can I have that library be consumable in a way that allows people to import the java class? I know it the .java file needs to get compiled into a .class file at some point, which makes sense in an uberjar, but what about pulling in my library via git or :local/root? Would it work / is it advisiable to put the .class file into version control? #2018-10-05 18:15:06 alexmiller You can’t - you should use a build tool to create and deploy an artifact #2018-10-05 20:29:37 nwjsmith I’ve been playing around with tools.deps using Maven as a build tool, and it’s been a good experience. clj -Spom -Srepro does mess with my pom.xml in undesirable ways, but testing is straightforward with the clojure-maven-plugin, and building uberjars with maven-shade-plugin works great. The most pleasant thing for me has been the speed relative to Leiningen, it’s much faster. #2018-10-05 20:30:15 nwjsmith I’m not sure why I haven’t seen this set up more often in the wild #2018-10-05 20:30:25 nwjsmith Fear of XML maybe? #2018-10-08 10:34:23 isaac clojure cli don’t care -Dmaven.repo.local? #2018-10-08 11:57:06 eval2020 @isaac you can add :mvn/local-repo "some/path" to deps.edn. #2018-10-08 12:15:38 isaac Is the :mvn/local-repo will be instead of “~/.m2”? #2018-10-08 12:25:36 eval2020 the provided path will be used instead of ~/.m2/repository #2018-10-09 05:00:49 Michael Fiano Hello all. I just noticed that -A must be supplied before the -r argument for them to take effect. Is that intentional or a bug? #2018-10-09 05:01:43 Michael Fiano Cancel that. I see in the usage notes that is intended #2018-10-09 23:35:39 kenny Getting UnknownHostKey: . RSA key fingerprint is when running clj -Spath. Any ideas? #2018-10-09 23:37:45 kenny Ah needed to add github to known_hosts. That should produce a better error message. #2018-10-10 05:43:30 dominicm @kenny the issue is because of ecdsa I believe. #2018-10-10 08:39:02 henryw374 It's a bit frustrating to have to wait so long for Windows support. https://dev.clojure.org/jira/browse/TDEPS-67 has been around for a while... Please add your vote! (not sure if it'll make a difference but it can't hurt). Thanks #2018-10-10 15:39:23 dominicm We're developing a situation where a few aliases have been split up in the name of DRY. This means that to start the project in "dev mode" requires a difficult number of aliases. Has anyone tried tackling this yet?
I'm currently thinking of writing something that merges together aliases so you can create a "super-alias" that's easier to maintain. #2018-10-11 09:11:56 Andreas Liljeqvist We use makefiles to handle starting tricky combos #2018-10-11 09:13:00 dominicm That doesn't go very well with cider-jack-in though, or any other editor integration? #2018-10-11 09:17:04 Andreas Liljeqvist You are right, we use it more for building releases and such #2018-10-11 09:17:15 Andreas Liljeqvist Don't have a good solution for launching dev #2018-10-10 15:40:31 dominicm This is really just a workaround for the lack of ability for aliases to depend on one another. #2018-10-10 15:41:14 seancorfield @dominicm Yeah, we're running into that a bit as we convert our Boot-powered monorepo to clj/`deps.edn`... #2018-10-10 16:38:37 zane I'd love to hear more about this if you'd like to share! Why you're doing it, etc. #2018-10-10 17:35:48 seancorfield @U050CT4HR We've had our dependencies in external EDN files for ages (years, maybe) and if fact called them deps.edn until clj/`tools.deps` appeared 🙂 I'm running into problems with Boot due to the fileset and pod stuff (I think because our project is large -- 83K lines across 400+ files -- but we haven't managed to tie down a repro case yet). Several of our Boot tasks take a long time to fire up and run, whereas running similar tasks via clj is much faster (since there's a lot less machinery involved). #2018-10-10 17:36:30 seancorfield I'm also a big fan of keeping close to the bleeding edge with Clojure releases and "core" tooling so having fewer dependencies in our system would make me happy. #2018-10-10 20:00:12 zane That's really helpful Sean. Thanks so much! #2018-10-10 15:41:48 dominicm @seancorfield have you had any particular ideas on how to tackle it? I'm quite happy to open source my solution. It will likely be a key piece in edge. #2018-10-10 15:42:14 alexmiller have you considered just repeating yourself? :) #2018-10-10 15:45:37 dominicm @alexmiller In some cases, yep! For a single dependency it's usually not so bad. When it's 2 or 3 aliases with 2 or 3 changes, the maintenance burden of bumping in multiple places is failing quite often. I'll admit that I'm trying to add in more "easy" onto this design than usual, as I can't impose pull request standards onto all projects. #2018-10-10 15:48:20 seancorfield We've had to repeat a chunk of our testing context in two separate aliases, which is a bit annoying. The alternative is a longer string of aliases. Right now we have a dummy project containing a deps.edn with mostly aliases. That has a :default aliases which has all our :override-deps where we "pin" versions of libraries. Then we have a deps.edn in each project that specifies the deps for that project (mostly as some/lib {} since the actual version comes from the :default alias) and references to other subprojects as worldsingles/something {:local/root "../something"}. This works pretty well for working within any given subproject and then we'll likely have a couple of shell scripts for running tests across the whole suite of projects. #2018-10-10 15:49:43 seancorfield Doing this has actually highlighted some problematic cross-project dependencies that Boot sort of hid for us, so that's been nice. And of course being able to use a much more lightweight process is always nice 🙂 #2018-10-10 15:49:56 dominicm heh, I actually just caught myself this second failing to update one of two places. I only caught it because I realized that a :main-opts looked out of place in the alias I had updated. #2018-10-10 15:53:15 seancorfield One thing I do find is that I'd desperately love to be able to parameterize deps.edn sometimes. For example, we can have an alias that starts an nREPL server to work with a project in an editor, but we can't affect which port it starts on -- that has to be hardcoded in the alias. So I'm thinking we may end up with one or more aliases that bring in (and run) certain small Clojure scripts that exist purely to support dev/test work. That way we could do clj -A:some:stuff:here <port-number> and run a -main in a dev namespace that starts nREPL on that port. #2018-10-10 15:53:42 seancorfield I'd be interested to hear more about how Cognitect folks deal with this in large projects. hint, hint 🙂 #2018-10-10 15:53:55 alexmiller don’t make large projects? #2018-10-10 15:54:02 seancorfield scowl #2018-10-10 15:54:43 seancorfield Is 63K lines of production Clojure and 20K lines of tests "large" tho'? #2018-10-10 15:56:13 alexmiller deps.edn being “just data” (and not having parameters, references, etc) has vast benefits for “simple” and is not something I will give up easily #2018-10-10 15:56:33 alexmiller at some point of complexity, you are really doing programming here #2018-10-10 15:56:46 seancorfield Oh, true. I totally get the benefits. I just haven't (yet) found the ideal balance yet, with the new tooling. #2018-10-10 15:56:56 alexmiller you can use clj to run programs - so pushing it into a script (which can take args) is one path #2018-10-10 15:56:59 dominicm @seancorfield I'm in favor of a main for the nREPL situation. #2018-10-10 15:57:14 alexmiller you can also wrap clj in another program (probably a shell script) #2018-10-10 15:57:22 alexmiller but could also be, say, make #2018-10-10 15:58:37 alexmiller make has the nice capability to define dependencies on smaller pieces which I think often is useful in build situations where you are making dependent subtasks #2018-10-10 15:58:42 alexmiller ditto ant #2018-10-10 15:58:46 seancorfield Being able to do something like this to run all subprojects tests is convenient enough: for d in * ; do (cd $d; CLJ_CONFIG=../versions clj -A:defaults:test:runner); done
#2018-10-10 15:59:12 seancorfield We're trying hard to get away from Ant. All that XML shudder #2018-10-10 15:59:46 alexmiller I don’t think running a standard “program” per project is any different though #2018-10-10 16:00:51 seancorfield Yeah, hence why I'm coming around to having a "dev" project with short Clojure scripts with -main entry points. And, to be honest, that's not hugely different from having tasks in a build.boot file which are all just functions. #2018-10-10 16:09:38 dominicm @seancorfield would the dev project be a dependency of all projects? #2018-10-10 16:13:59 seancorfield Only under an alias for dev/test tooling -- it wouldn't be a regular dependency. #2018-10-10 16:19:58 dominicm sorry, yeah. #2018-10-10 18:40:50 dominicm @seancorfield once add-lib is stable, it might be a good idea to dynamically fetch dependencies based on the task(s) being run. e.g. nrepl will grab cider/tools.nrepl on demand. That would probably reduce the number of aliases way down. Basically making most of the project static (deps.edn) but providing a dynamism for the non-simple stuff. #2018-10-16 15:01:53 sparkofreason What's add-lib? Sounds intriguing. #2018-10-16 19:36:23 dominicm It's a function in a branch of tools.deps #2018-10-10 18:42:56 seancorfield I dream of a world where tooling bootstraps itself on top of prepl instead of needing server-side dependencies 🙂 #2018-10-10 18:43:06 seancorfield (I love unravel for that reason) #2018-10-10 18:48:14 dominicm socket repl already does that, but I agree 🙂. But I think the same applies (for me) for loading in cljs to do a one-time build. #2018-10-10 18:55:52 seancorfield Well, specifically the tooling. I'd like to see a version of CIDER that could bootstrap itself over a socket REPL or prepl. So you didn't need all the CIDER deps on the project side of the equation. Same with ProtoREPL for Atom. #2018-10-10 18:57:27 seancorfield What I like about unravel is that I can spin up a bare Socket REPL (e.g., on a production JAR, via JVM options) and then connect via unravel and when I (require 'compliment.core) it sends it over the wire from the REPL so I don't need it in my project. Then I have a nice interactive REPL with history, documentation hinting, completion... #2018-10-10 19:00:22 seancorfield Your project then becomes tooling neutral. You don't need any dependencies in it to support tooling. And then you can use any tooling with it. I know CIDER can inject itself into a newly-started REPL process, but that's a heavyweight server containing all of those dependencies. I basically want CIDER-like functionality over a bare Socket REPL started from a JAR that doesn't have those dependencies 🙂 #2018-10-10 19:24:04 dominicm I just discovered I can use find-deps as a filter! Nice. This means I can do !aE in vim, followed by my find-deps command to auto-update the buffer I'm working on. Cool! #2018-10-10 19:29:03 dominicm @alexmiller have you had any thoughts about comments being obliterated by -Sresolve-tags? #2018-10-10 19:30:18 dominicm I should open a jira if there isn't one #2018-10-10 19:30:58 alexmiller reader doesn’t preserve comments #2018-10-10 19:31:28 alexmiller I’m not planning on doing any more work on resolve-tags. most likely will actually extract it all out into a separate project and remove it entirely #2018-10-10 19:33:42 alexmiller as an external program you could invoke it from clj if needed #2018-10-10 19:40:21 dominicm If it's an external program that shifts expectations I think, cool. I would be more inclined to reimplement it with rewrite-clj so I could preserve comments/formatting. #2018-10-10 19:47:31 dominicm I've been thinking about whether a wrapper over rewrite-clj that implements IAssoc is feasible. #2018-10-10 20:26:19 alexmiller Well feel free to rewrite it and make it do whatever :) #2018-10-10 22:11:20 seancorfield @arrdem yes I use that with socket server REPLs a lot. #2018-10-10 22:12:28 seancorfield It can bootstrap compliment into a running REPL to give you auto-completion, which I like. #2018-10-10 22:29:35 andy.fingerhut I guess bootstrap here just means "send the necessary text across the socket to define functions/etc. to create the necessary code on the server?" #2018-10-10 22:32:17 seancorfield Yeah. In this case, "send the Compliment library across the socket..." #2018-10-11 01:38:06 seancorfield ;; uberjar building:
;; - see for a minimalist approach:
;; - uses my fork that adds manifest support
;; - clj -A:depstar result.jar -m
;; - sets clojure.main-main as the entry point
;; - clj -A:depstar result.jar my.project
;; - sets my.project/-main as the entry point
:depstar {:extra-deps
{com.healthfinch/depstar
{:git/url ""
:sha "a328136da59ea9b83483189451fb71aafd13bdb1"}}
:main-opts ["-m" "hf.depstar.uberjar"]}
#2018-10-11 06:35:05 Michael Fiano As a beginner to tools.deps and Clojure in general, I just banged my head against the wall for a couple hours figuring this implementation detail out, because the documentation leaves off this edge case. I discovered something with alias loading I thought would be worthwhile to share here in any case:
Assuming you have 2 aliases with :main-opts keys, regardless if other keys from other alias types are present in them or not:
If -M is specified on the command line for both aliases, only the last alias is used as documented.
If -A is specified on the command line for both aliases, only the last alias is used as per the documentation for -M.
If -A is specified for one, and then -M is specified for the other, only the last one is used.
If -M is specified for one, and then -A is specified for the other, still only the -M one is used, even though -A is later.
It's that last rule that got me good. In short, you have to be very careful here. #2018-10-11 08:12:55 martinklepsch Anyone recommendations for a thing that will package my deps.edn lib into a jar and install it into the local repo? #2018-10-11 08:16:23 martinklepsch Pack doesn't do regular jars, cambada doesn't install them into local maven, meyvn seems a little more involved #2018-10-11 08:33:26 orestis I think the other one is depstar? No idea if it does what you want though. #2018-10-11 08:46:27 mpenet yeah they all are a bit tricky to use, each with their own set of tradeoffs #2018-10-11 08:48:08 mpenet personnaly I am half tempted to just rely on having clj installed and run apps with it, more or less what you do with cljdocs I think @martinklepsch right (bash script that downloads the archive, upacks it and runs)? #2018-10-11 08:51:58 mpenet capsule based build seemed quite a good idea (one of the options on juxt.pack), but I am not sure where that project is heading #2018-10-11 09:02:31 dominicm @mpenet happy to answer questions you might have! #2018-10-11 09:05:42 martinklepsch I'm actually referring to regular jars (not uberjars)... e.g. when publishing a library #2018-10-11 09:13:29 dominicm @martinklepsch fwiw, you could use pack with skinny jar mode and write a pom.xml. But you'd have to install that jar yourself (with maven I guess?) #2018-10-11 11:25:40 mpenet @dominicm nothing special in mind. pack is imho the most complete solution so far. One minor missing piece would be the ability to specify a main on the one-jar option, but i can live without it #2018-10-11 16:24:18 dominicm There should be an issue open for this. It is something I want to fix. I don't want to require AOT from users, so I want to patch onejar a little to support a default param to clojure.main. #2018-10-11 12:23:07 orestis Would be nice if there was only one well supported solution for packaging. I stared with cambada but had to fork to apply some PRs #2018-10-11 14:50:15 seancorfield @orestis sounds like me forking depstar to apply a PR to add a minimal manifest 😀 #2018-10-11 14:51:31 seancorfield @martinklepsch is there a reason you need a jar in your local repo? Can't you refer to that local library source via :local/root? #2018-10-11 15:07:58 martinklepsch @seancorfield I can but I have some wierd interdependencies that aren't supported fully by Cursive/tools.deps #2018-10-11 15:08:49 cfleming Right, transient :local/root deps are not well supported (by deps itself, and also Cursive - I’m fixing that right now) #2018-10-11 15:12:43 orestis Even though I don’t use Clojure professionally yet, I’d chip in for an Arachne-like Kickstarter, but for a new build tool based on tools.deps and simple libraries. #2018-10-11 15:13:43 orestis There’s so much duplication of effort these days - jar building, linters, formatters and so on. #2018-10-11 15:15:03 orestis I have a dream for a build tool that would just provide an (overridable/extendable) set of library decisions and some glue code to invoke them from the command line. #2018-10-11 15:28:02 Andreas Liljeqvist What you are describing is probably a program 😕 #2018-10-11 15:31:15 Andreas Liljeqvist But yeah, there is a lot of churn in the dev space right now #2018-10-11 15:31:41 Andreas Liljeqvist Will probably be a while before it simmers down with some kind of best practices #2018-10-11 15:33:31 orestis The whole point is to rally everyone around a single tool. I think this has multiplier benefits for the community and newcomers. See for example Elixir and their mix tool. #2018-10-11 15:40:18 Andreas Liljeqvist There is certainly value for newcomers. #2018-10-11 15:40:56 Andreas Liljeqvist Atm Lein is probably good for them #2018-10-11 15:41:45 Andreas Liljeqvist But tools-deps is the first system that let me understand wtf was going on when starting up #2018-10-11 15:41:51 Andreas Liljeqvist sooo.. #2018-10-11 15:43:15 orestis Yeah, I’m all for tools.deps for dependencies and classpath stuff. #2018-10-11 16:28:02 seancorfield I've been recommending clj to newcomers -- with the caveat that most books/tutorials use lein. At least with clj, you can point newcomers at "official" documentation in a central place and you know the tooling is both simple and well-supported. #2018-10-11 16:34:12 orestis As a relative newcomer, clj is just so much less cognitive overhead than Lein. #2018-10-11 15:34:25 seancorfield For a long time, at work we've done no AOT and we build over uberjars to have clojure.main as an entry point. Our needs are very simple. We used to run a lot of stuff from source directly via lein and later boot but I think packaging as an uberjar for java -jar is simpler and safer (no need to fetch dependencies onto the target systems). #2018-10-11 15:38:12 Andreas Liljeqvist +1 for building uberjars #2018-10-11 15:35:14 seancorfield ClojureScript has had years of churn but seems to be settling down now. Maybe Clojure needs a bit of churn to shake things out and give us better, simpler tools too? #2018-10-11 15:36:23 Andreas Liljeqvist tools-deps is a great step in that direction. But causes a bit of chaos in the transition #2018-10-11 15:46:06 orestis To be clear, IMO tools-deps enables the creation of such a mostly-glue tool. Otherwise making a new build tool from scratch is a huge undertaking. #2018-10-11 16:05:42 arrdem @martinklepsch the official story there seems to be write a pom.xml yourself, use the -Spom task to keep the dep lists in sync? (I don’t really understand what it does) and mvn install yourself. #2018-10-11 16:26:05 dominicm Having attempted to write that build tool, I'm not happy with anything I've come up with. Build tooling is hard to get right, especially around tools like figwheel. #2018-10-11 16:35:17 orestis Share notes perhaps? Any attempt to explore the space is very valuable. #2018-10-11 16:36:17 dominicm Krei.alpha and kick.alpha are the two attempts by JUXT. We also did mach in the past which we used for alternative purposes. #2018-10-11 16:38:08 dominicm @U064X3EF3 keeps mentioning that builds are programs, so I suspect that both of those attempts to be data driven are fatally flawed.
I've been thinking of implement a paper by peyton jones for creating build systems a la carte, maybe that will unlock the pieces needed, but I'm not sure yet. #2018-10-11 16:55:06 orestis My naive attempt so far is to just invoke some namespaces - either external or my own via aliases. My need is for more well rounded and robust libraries to do things like jars and so on. #2018-10-11 16:57:36 dominicm Caching. Minimal builds. Early cut off. These are the general problems to solve. Cloud building is a nice to have in some contexts, but none I'm in. #2018-10-11 17:01:52 orestis I’ll look at krei and kick, thanks. #2018-10-11 17:05:32 orestis I don’t think that there is any way around the current slow startup of clojure, is it? Even with precompiled stuff, you will get at least a handful of seconds which is already too much. Hard stuff. #2018-10-11 17:09:46 rickmoynihan you could always fire up a build-tooling repl once, that puts you in a namespace with functions like (build) (test) (do-everything) #2018-10-11 17:16:40 zane @orestis Have you seen the results people are getting with GraalVM? #2018-10-13 08:54:24 dominicm Something quite hard to build at a REPL is AOT-compiling namespaces. You have to start a new JVM to do that. You might be able to get away with something like a pod, but that's the best I have. Just as a result of how it decides to cache/skip namespaces. #2018-10-11 16:31:03 lilactown especially since the UX is so dependent on perf. everyone wants things to be fast #2018-10-11 16:31:07 lilactown or at least, fail fast #2018-10-11 16:39:12 dominicm To be honest, I think you have to just make it a library that you can run from the repl. It has to happen around the Clojure command line tools, else there is too much friction. #2018-10-11 16:42:02 dominicm Even shell scripts get in the way of tooling. Cursive won't work if you require funk. #2018-10-11 16:42:41 seancorfield I really do like depstar's minimalism but it's frustrating that it omitted a minimal manifest -- and I'm not sure why it did that. Everything else in the build space is much more complex 😐 #2018-10-11 16:44:26 dominicm @seancorfield the easy answer is that generating manifests is out of scope. War is another format that needs a manifest to be written. Simpler to let users decide how to handle extra files in their classpath. #2018-10-11 16:44:50 dominicm I don't mean to speak for the author, just a thought I've had working on pack. #2018-10-11 17:04:48 rickmoynihan yes, not so minimal when you start having to do that… #2018-10-11 17:09:55 seancorfield Adding a manifest with just a main class is only a few lines of Clojure when you're building a JAR tho'... 🙂 #2018-10-11 17:28:24 dominicm But it conflicts with any additional things a user may want to insert into a manifest themselves. Users can also generate a manifest in a few lines of code. In most cases of can just be hardcoded by the user. #2018-10-11 18:47:34 seancorfield Right, which is why it's important that adding the minimal manifest is optional. depstar specifically excludes META-INF/*.MF files from the classpath so the resulting JAR is guaranteed to have no manifest. My fork allows users to conditionally add a minimal manifest that specifies Main-Class (only). That seems to be such a common need -- for java -jar ... -- that baking it in as an option seems worthwhile. #2018-10-11 18:48:31 seancorfield The code is based on Boot's create-manifest so it could also take a hash map of key/value pairs to add as extra manifest attributes, but that's not exposed to the command-line in my fork (yet). #2018-10-11 18:48:35 dominicm I see. That seems quite wrong. It also removes license files which is not allowed under some licenses like Apache #2018-10-11 18:49:22 dominicm Fun fact: run manifest doesn't capture the full capabilities of manifests! #2018-10-11 18:49:37 seancorfield Yeah, I noticed that and thought it was weird. I don't know why it does that (and Ghadi hasn't responded to any of my questions about it yet). #2018-10-11 18:51:47 seancorfield We're not redistributing our JARs so the license issue isn't a problem -- and depstar filters out a lot of other stuff too so it's pretty brutal in that respect. #2018-10-11 18:35:29 arrdem Is there an algorithmic reason beyond wanting to track the path for every expanded dep that expand-deps couldn’t group deps into resolution batches by manifest type? The Maven implementation used to read manifests seems designed for reading many manifests in one go, and less so for reading a single manifest at a time. #2018-10-11 21:03:11 arrdem The hard thing is proving that letting Maven potentially solve range constraints instead of using deps’ behavior for everything can’t produce a different final dependency solution 😕 #2018-10-11 22:19:26 alexmiller Maven will produce a different solution, and we don’t want it #2018-10-11 23:28:47 arrdem Ah. Yeah if you have a non-Maven pinning (or even just winning coordinate) for a dep, and Maven resolves something else, you can’t really back the transitive consequences of that out of Maven’s solution so you loose minimality for sure which can have other consequences. #2018-10-11 19:11:35 Michael Fiano Regarding my message above about aliases, does anyone think this deserves an issue opened to at the very least document this edge case? #2018-10-11 19:15:20 arrdem The order independence of -M winning when -A and -M is present is probably worth documenting. #2018-10-11 19:15:25 arrdem So yes IMO. #2018-10-11 19:20:30 dominicm I would prefer to see it fixed, but documentation would be an improvement over the current situation. #2018-10-11 19:23:30 Michael Fiano Personally I'd like to see the behavior change. The reason why is this: #2018-10-11 19:23:41 Michael Fiano :
I have a :dev alias that is only ever project-local. I use it for example to require my namespace when starting up an nrepl in cider. This requires a :main-opts key, making it of the -M type, but as per the documentation, only the last -M type is ever used. I also want to compose this with a debugging alias to say depend on and require spyscope with another alias. Currently this is impossible without putting that in the :dev alias and thus copying it to be project-local instead of a global alias. #2018-10-11 19:24:00 Michael Fiano Unless there is some other way to compose a global main-opts with a project local one that I do not know of #2018-10-11 19:25:04 Michael Fiano Maybe I'm just asking too much from this too new tool? #2018-10-11 19:28:53 seancorfield FWIW, I only ever use -A and I split aliases out so I run into this issue (downside: you need to specify multiple aliases for some things). #2018-10-11 19:29:25 Michael Fiano Perhaps @U064X3EF3 can weigh in here as I see he is the project lead. When i get home later I'll open an issue if only to document this edge case so other newcomers don't bang their heads like I did. #2018-10-11 19:33:16 andy.fingerhut It is better to ask about whether it can change when things are still alpha, than later. It is really up to folks like Alex, Rich, and Stu Halloway on how they choose to make the tool work, so don't be surprised with the answer "doc enhancement, please" #2018-10-11 19:35:22 seancorfield My sense is that it is "by design" that aliases are combined in order, left-to-right and that -A:foo is just shorthand for -R:foo -C:foo -M:foo ... -- so the behavior you observe is intentional and likely considered to be already well-documented. #2018-10-11 19:36:58 seancorfield If you specify your project-local alias with -R and/or -C instead of -A then you'll get the behavior you want. #2018-10-11 19:37:03 Michael Fiano The confusing bit is that -M is documented as only the last one being used, but if -A is indeed shorthand for that, an -A after an -M needs to have a note mentioning that only the last explicit -M is applied, and -A is ignored. #2018-10-11 19:37:47 seancorfield The -A is not ignored -- the aliases are all combined left-to-right, but with :main-opts it's always last one wins. #2018-10-11 19:37:58 Michael Fiano But that is not the case. #2018-10-11 19:38:29 Michael Fiano -M:foo -A:bar, when both foo and bar have :main-opts keys, then foo wins. #2018-10-11 19:42:21 seancorfield Ah, gotcha. Here's why: val="$(join '' ${resolve_aliases[@]})|$(join '' ${classpath_aliases[@]})|$(join '' ${all_aliases[@]})|$(join '' ${jvm_aliases[@]})|$(join '' ${main_aliases[@]})|$deps_data"
#2018-10-11 19:43:21 seancorfield So -M is always applied last. OK, yes, that should be documented. Do you want to create a JIRA ticket for that (against the CLJ JIRA project I believe)? #2018-10-11 19:43:57 seancorfield I'm happy to create the ticket if you want me to. #2018-10-11 19:44:05 Michael Fiano I very much want to so others don't run into the headaches I did. I won't be able to do that until tonight or tomorrow though when I'm at my dev machine, unless you don't mind. #2018-10-11 19:46:22 seancorfield I'll go ahead and create it... against the website since it's really an error in the Reference page for the CLI #2018-10-11 19:46:39 Michael Fiano Thanks @seancorfield! #2018-10-11 22:00:32 alexmiller This is a bug, not a doc issue. Order matters, should be respected. #2018-10-11 23:33:13 seancorfield OK, good to know. So the shell script would need to expand each -A as it encountered it, adding the aliases as if all four were really specified explicitly? #2018-10-11 23:33:52 seancorfield Do you want a JIRA issue (and maybe a patch) against CLJ for that instead? #2018-10-11 23:46:01 alexmiller Jira yes, patch if you like. I suspect it affects the scripts too so may require some coordination #2018-10-11 23:52:28 seancorfield Ugh! Yes, fixing the clojure script would solve one side of things -- that's easy -- but tools.deps also needs to be fixed too. OK. JIRA for now. I'll look at a patch if I get enough free time but it's a bigger change than I had initially thought. #2018-10-12 01:42:16 Michael Fiano Thank you everyone. #2018-10-12 01:37:19 seancorfield Yeah, that is a lot more comprehensive but has a lot of knobs and dials -- and has its own set of surprises, I gather, from people who've used it. #2018-10-12 06:03:24 tianshu is that in tools.deps, there's no exclusions? #2018-10-12 06:08:09 seancorfield @doglooksgood :exclusions work fine in deps.edn #2018-10-12 06:11:02 seancorfield Do you have a specific case where you think it doesn't work? #2018-10-12 06:13:46 dominicm @doglooksgood there isn't a top level exclusions though, which is something people try though, only a per-dependency exclusions. #2018-10-12 06:15:03 tianshu @seancorfield When using leiningen, usually I found there're some conflicts in my dependencies, but with clojure -Stree, I don't get any information like this. #2018-10-12 06:15:22 tianshu So I don't know if there's conflicts or not #2018-10-12 06:18:17 dominicm @doglooksgood I think tools.deps operates on a different principle to maven/lein, so conflicts are less of an issue. tools.deps always picks the newest version, as opposed to maven which picks the first one it finds. #2018-10-12 06:24:12 tianshu SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/tianshu/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j
/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/tianshu/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2
.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
I want to remove one of the binding, but don't know where introduce the multiple bindings. #2018-10-12 06:27:35 tianshu with lein I got some information like this:
Possibly confusing dependencies found:
[org.quartz-scheduler/quartz "2.3.0"] -> [org.slf4j/slf4j-api "1.7.7"]
overrides
[conman "0.8.1"] -> [hikari-cp "2.4.0"] -> [com.zaxxer/HikariCP "3.1.0"] -> [org.slf4j/slf4j-api "1.7.25"]
and
[org.quartz-scheduler/quartz "2.3.0"] -> [com.zaxxer/HikariCP-java6 "2.3.13"] -> [org.slf4j/slf4j-api "1.7.10"]
and
[ch.qos.logback/logback-classic "1.2.3"] -> [org.slf4j/slf4j-api "1.7.25"]
Consider using these exclusions:
[conman "0.8.1" :exclusions [org.slf4j/slf4j-api]]
[org.quartz-scheduler/quartz "2.3.0" :exclusions [org.slf4j/slf4j-api]]
[ch.qos.logback/logback-classic "1.2.3" :exclusions [org.slf4j/slf4j-api]]
#2018-10-12 06:28:40 tianshu then I know I can remove the binding by add exclusions on the org.quartz-scheduler/quartz dependency. #2018-10-12 06:30:37 dominicm @doglooksgood this is different to conflicting dependencies. clj -Stree gives you the information you need to fix this, you need to find which dependency is bringing in the logback you don't want, either nop of logback-classic. #2018-10-12 12:25:46 alexmiller I don’t think -Stree actually does provide info to do that - it just lists what is used, not what isn’t #2018-10-12 12:25:59 alexmiller This is an area that needs some more work #2018-10-12 12:26:46 alexmiller If you don’t like the one you’re getting, you can try excluding that one and see which one you get instead #2018-10-12 14:57:56 dominicm @alexmiller that's what I meant. It will show where logback-nop comes from. #2018-10-12 16:02:05 seancorfield @alexmiller Any thoughts on adding some sort of "pedantic" check to tools.deps? #2018-10-12 18:04:24 alexmiller in terms of reporting or also failing on pedanticness? #2018-10-12 18:18:48 Michael Fiano Are there any plans to allow whitespace in strings? #2018-10-12 18:19:38 Michael Fiano I was reading the edn spec and it says commas are not treated as whitespace within strings, but I must use them for arbitrary code using -e for example #2018-10-12 18:36:06 alexmiller would like to allow, in practice it’s painful due to the interaction with the shell #2018-10-12 18:37:58 Michael Fiano I see. Editor word-wise commands etc get confused so it's painful to edit #2018-10-14 10:05:13 isaac does deps.edn supports set env or properties? #2018-10-14 10:41:56 dominicm @isaac Yes. Although I'm not certain that I fully understand what you're asking. #2018-10-14 10:43:04 dominicm e.g.
❯ clj -J-Dx=y -e '(System/getProperty "x")'
"y"
works. #2018-10-14 10:48:18 isaac :local/root "${LIB_ROOT}/xxx.jar" #2018-10-14 10:48:58 dominicm @isaac no, this doesn't work. #2018-10-14 10:51:38 isaac there have difference normally between local dev env and production env #2018-10-14 10:53:04 isaac IMO, deps.edn supports env variable (or property) is necessary. #2018-10-14 10:54:20 dominicm You can emulate this by using -Sdeps. e.g. clj -Sdeps "{:deps {foo/bar {:local/root \"${LIB_ROOT}/xxx.jar\"}}}" #2018-10-14 10:54:50 dominicm Normally we solve this by using different aliases for dev and production. #2018-10-14 11:17:21 isaac why :main-opts ["-e" "(println 'hello)"] throws EOF while reading? #2018-10-14 11:24:09 isaac it seems the :main-opts splits arguments by space #2018-10-14 12:23:01 mfikes Workaround with Corfield comma:
:main-opts ["-e" "(println,'hello)"]
#2018-10-14 12:50:58 aisamu >Corfield comma
🙂 #2018-10-14 19:25:34 dominicm Your vector is just a string #2018-10-14 19:27:05 dominicm You may be able to compose them, but I don't think so. #2018-10-14 19:27:32 dominicm I think the correct fix is multiple -m flags #2018-10-14 20:25:55 richiardiandrea Yep i agree! See suggestion in the nRepl issue 😄 #2018-10-15 10:10:57 dominicm Unexpected side effect of gitlibs with "optional" namespaces, e.g. for myapp.ns.spec. tools.namespace will try and load the namespace regardless. This is also frustrating for :local/root dependencies, where I actually want to be able to refresh and work on both in parallel, but not necessarily on the specs. #2018-10-15 10:27:47 dominicm This is similar to what clojure.core goes through I suppose. #2018-10-15 15:31:22 danielneal I'm doing something derp-y here - can anyone halp - what have I missed out?
clj -Sdeps '{cheshire {:mvn/version "5.8.1"}}'
Clojure 1.9.0
user=> (require '[cheshire.core :as json])
FileNotFoundException Could not locate cheshire/core__init.class or cheshire/core.clj on classpath. clojure.lang.RT.load (RT.java:463) #2018-10-15 15:32:13 alexmiller clj -Sdeps '{:deps {cheshire {:mvn/version "5.8.1"}}}' #2018-10-15 15:32:20 alexmiller ^^ missed :deps layer #2018-10-15 15:32:21 danielneal ah awesome! Thanks #2018-10-15 15:32:37 danielneal so excited using the cli for the first time #2018-10-15 15:33:02 alexmiller I’ve been thinking about adding a warning as this is the nth time someone has asked a question like this #2018-10-15 15:33:12 alexmiller (inc n) #2018-10-15 15:33:37 danielneal haha so yup I can't read but at least it's not just me #2018-10-15 15:33:54 danielneal warning++ #2018-10-15 15:34:54 danielneal so nice just to get a quick clojure repl to prod this json object - I never can remember how to get jq to do what I want. #2018-10-15 16:01:45 seancorfield Yeah, I've been using clj for ages and I still do this, omitting the :deps layer! Probably at least once a day. #2018-10-15 16:15:08 mfikes Another warning that could prove useful is if you omit the :mvn/version layer as in
{:deps {cheshire "5.8.1"}}
I do this, say, once per week. #2018-10-15 16:15:56 seancorfield You do get an error for that -- but it's a bit obscure. #2018-10-15 16:16:11 mfikes Ahh, cool #2018-10-15 16:16:47 seancorfield Error building classpath. Coordinate type not loaded for library cheshire/cheshire in coordinate "5.8.1"
#2018-10-15 16:17:14 seancorfield (and, yeah, I do that all the time too!) #2018-10-15 16:20:35 orestis Sounds like a need for some kind of tool that allows you to write specifications for data? #2018-10-15 16:20:52 mfikes Yeah, one that is in alpha 🙂 #2018-10-15 16:22:26 orestis I’ve never typoed the maven thing, but it could be I’m just editing an existing file with plenty of examples. #2018-10-15 16:36:57 andy.fingerhut or perhaps a lint-like tool that has custom non-spec code that looks for and warns about data structures that look like they may be mistakes... #2018-10-15 16:51:47 seancorfield We've made a lot of progress in this last week on moving from Boot to clj/`deps.edn` for our monorepo. #2018-10-15 16:52:58 seancorfield The issues with relative paths in :local/root has meant we have to have everything "side by side" in one part of the repo. #2018-10-15 16:53:38 seancorfield And we found we really needed a baseline deps.edn (in the project) and a per-subproject deps.edn (for each subproject in the repo). #2018-10-15 16:54:42 seancorfield So we ended up having a dummy subproject and pointing CLJ_CONFIG at that folder, so we can have all our "pinned" versions in :override-deps in an alias that every command has to use CLJ_CONFIG=../versions clj -A:defaults:other:aliases
#2018-10-15 16:55:34 seancorfield We have a small "build" tools subproject with a bunch of -main entry points for various tasks and we bring it in with aliases (in ../versions/deps.edn). #2018-10-15 16:58:24 seancorfield We'll probably have a couple of shell scripts to provide some sugar over those invocations but we've got nREPL capabilities, test, build a JAR, run migrations (SQL/data) all up and running with "just" clj and a few aliases. I need to convert two more pieces this week and we'll be close to just removing Boot from our pipeline. #2018-10-15 17:17:51 lwhorton hey with the new tools.deps i’m tooling around with a local jar file via the :local/root "/path/to/jar", but changes to the jar don’t seem to be reflected in the repl session. what do i have to do to (reload) or re-require after changing the jar so that the latest code is around in the repl? #2018-10-15 19:13:18 alexmiller I’m not sure that you can get the default Java classloaders to do this - they don’t pick up modified jar files after startup. So even reloading your Clojure namespace is probably not going to help afaik. #2018-10-15 19:30:25 lwhorton that seems to be what’s happening. for now we’re just hard-restarting the whole repl #2018-10-15 20:15:56 alexmiller Something like a boot pod would give you a fresh classloader and the ability to do this #2018-10-15 17:29:18 seancorfield That sounds like a plain ol' REPL workflow issue and you'd need to tell Clojure to reload namespaces from that JAR, regardless of which tool you were using. #2018-10-15 17:29:55 seancorfield Mind you, if you're rebuilding the JAR locally, why not point :local/root at the source code instead and remove the JAR step? #2018-10-15 17:30:33 seancorfield (you'll still need to tell Clojure to reload the code -- after all, the classpath is not changing and Clojure doesn't know to re-read files on the classpath unless you tell it to) #2018-10-15 17:33:38 lwhorton > reload namespaces from that JAR
is that a good old (require '[foo.bar]) again? #2018-10-15 17:56:33 mfikes @lwhorton (require '[foo.bar] :reload) #2018-10-15 18:02:43 dominicm @seancorfield what layout did you have before? #2018-10-15 18:13:58 seancorfield @dominicm We had a top-level deps.edn (equivalent) and our misc. build tooling was in a completely separate folder build/
<tooling here>
clojure/
deps.edn equivalent
subproject-a/
deps.edn
subproject-b/
deps.edn
... #2018-10-15 18:17:12 seancorfield Now we have build/
<wrapper shell script>
clojure/
build/
deps.edn
build/dev-related Clojure code
versions/
deps.edn -- pinned versions and aliases
subproject-a/
deps.edn
subproject-b/
deps.edn #2018-10-15 18:19:00 dominicm @seancorfield not super different then. Interesting solution using CLJ_CONFIG. Hard to get people to set that up, but not a problem in a smaller team though. #2018-10-15 18:20:09 seancorfield That's why we have a wrapper shell script 🙂 #2018-10-15 18:23:48 seancorfield The shell script is basically build task subproject and it maps mostly onto (cd path/to/<subproject>; CLJ_CONFIG=../versions clojure -A:defaults:<task>)
#2018-10-15 18:24:35 seancorfield build nrepl datamapper or build test environment or build migrate sql etc #2018-10-15 18:43:28 dominicm @seancorfield that doesn't seem very amenable to cursive or ciders jack in? How are you editing? #2018-10-15 19:14:52 seancorfield Well, I'm using Atom/ProtoREPL which doesn't support clj in any useful way and my colleague who uses Emacs/CIDER always starts his REPLs by hand and then connects to it via CIDER... #2018-10-15 19:15:49 seancorfield Because of the whole CLJ_CONFIG=../versions clojure -A:defaults thing, it's a pain to configure tools to start clj-repls that way anyway. #2018-10-15 19:33:11 dominicm Yeah. It is as a result of that 🙂. You could restore that functionality by automating some kind of alias merging across your deps.edn files. #2018-10-15 19:33:31 dominicm (This is something I expect I will end up writing, but I'm still fiddling with ways to avoid it) #2018-10-15 19:47:08 seancorfield You'd either need to have support for multiple deps.edn files in a clj command or your user deps.edn file would need everything we have in versions/deps.edn tho'... (if you mean to remove CLJ_CONFIG=../versions from the command itself) #2018-10-15 19:47:21 seancorfield Either way, not a problem for the sort of usage we have/need. #2018-10-15 19:47:56 dominicm I understand. I was looking to you for a silver bullet, as I'm currently getting Edge more & more tools.depsy. #2018-10-15 19:48:19 seancorfield There are no silver bullets (tm)! #2018-10-15 19:48:58 dominicm I can feel Fred Brooks channeling through you #2018-10-15 20:40:58 andy.fingerhut What are your views on bronze bullets? 🙂 #2018-10-15 21:02:00 arrdem @seancorfield I’d be curious what you think of https://github.com/arrdem/katamari - I just got it to the point of being able to build classpaths, uberjar and self-host last night. It’s designed to solve the sorts of many targets and shared configuration in a single monorepo friction you’re encountering as I hit them too in my first pass at deps for our monorepo. #2018-10-15 21:34:06 zane 🎶 Na na~ 🎶 #2018-10-15 21:08:06 seancorfield Will take a look. Thanks @arrdem #2018-10-15 21:19:53 arrdem It’s extremely pre-alpha and currently extremely under-documented, but I’m happy to answer questions and will be trying to clean it up as I go. #2018-10-16 02:03:27 kenny I have this code:
(ns where-print
(:require
[clojure.spec.alpha :as s]))
(s/def ::foo string?)
(defn -main
[& args]
(when-not (s/valid? ::foo 1)
(s/explain ::foo 1)
(System/exit 1)))
If I run clj -m where-print nothing is printed to stdout. Any idea how to prevent this from happening? #2018-10-16 02:45:21 seancorfield (System/exit 1) is a pretty brutal way to exit the program and it can quit the JVM before output is flushed. If you add (flush) before that exit, you should get your output. #2018-10-16 02:49:47 seancorfield It would also work if you did this: (defn -main
[& args]
(when-not (s/valid? ::foo 1)
(println (s/explain-str ::foo 1))
(System/exit 1))) since println will flush stdout to the newline (although your output will now have two newlines) @kenny #2018-10-16 03:07:56 kenny @seancorfield thanks. By brutal, are you hinting at a better way to exit the process with a unsuccessful exit code? #2018-10-16 03:14:51 seancorfield No. Just observing that it it immediately kicks off a shutdown of the JVM and some things may not get done that you are expecting 🙂 #2018-10-16 06:01:02 dottedmag JVM — Unleash your inner Hulk! #2018-10-17 07:37:42 dominicm Similar data point to knowing about conflicts, I'd really like to know if a dependency wants a newer version of a dependency than I am giving it.
project depends on b 0.1
project depends on a
a depends on b 0.2 #2018-10-17 18:55:54 seancorfield As of this morning, our entire CI pipeline (including SQL migrations, JAR building and deployment to servers) runs on clj/`deps.edn` for most of our projects -- previously that was all on Boot. #2018-10-18 09:03:04 martinklepsch do you also build jars for libraries which you push to a maven repo? I find the existing libs mostly focus on uberjars or don't help with the creation of the pom file/actual deployment. #2018-10-18 14:49:20 seancorfield We only create uberjars for deployment. #2018-10-17 20:10:03 ghadi dang! #2018-10-17 20:20:20 alexmiller ! #2018-10-17 20:21:00 alexmiller Mostly a fix/recommendation around connecting to private s3 maven repos #2018-10-18 11:55:04 pauld How can I debug the clj tool chain? clj hangs when I put org.openjfx/javafx-graphics {:mvn/version "11"} in the deps.edn file. #2018-10-18 11:59:28 gfredericks did it successfully download the jar? #2018-10-18 12:02:20 bronsa I see a massive cpu spike after clj finishes downloading the artifact #2018-10-18 12:04:09 pauld I don't know. I can get it working it boot, so boot was able to download the jar. #2018-10-18 12:04:52 pauld maybe I need to wait longer? Still it works just fine in boot, so there is some issue here with the clj tool. #2018-10-18 12:09:45 bronsa it looks to me like there's some infinite loop during dependency resolution #2018-10-18 12:10:10 mpenet The latest clj should fix that #2018-10-18 12:10:26 mpenet At least there was an issue related to this #2018-10-18 12:10:42 bronsa yeah I remember that, but I'm on latest tools.deps #2018-10-18 12:10:50 mpenet TDEPS-79 #2018-10-18 12:10:56 mpenet oh #2018-10-18 12:10:56 bronsa at clojure.lang.APersistentVector.hasheq(APersistentVector.java:162)
at clojure.lang.Util.dohasheq(Util.java:177)
at clojure.lang.Util.hasheq(Util.java:168)
at clojure.lang.PersistentHashMap.hash(PersistentHashMap.java:120)
at clojure.lang.PersistentHashMap.valAt(PersistentHashMap.java:152)
at clojure.lang.PersistentHashMap.valAt(PersistentHashMap.java:156)
at clojure.lang.RT.get(RT.java:753)
at clojure.core$get.invokeStatic(core.clj:1492)
at clojure.core$get.invoke(core.clj:1486)
at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58)
at clojure.core$reduce1.invokeStatic(core.clj:934)
at clojure.core$get_in.invokeStatic(core.clj:6062)
at clojure.core$get_in.invoke(core.clj:6062)
at clojure.tools.deps.alpha$excluded_QMARK_.invokeStatic(alpha.clj:87)
at clojure.tools.deps.alpha$excluded_QMARK_.invoke(alpha.clj:83)
at clojure.tools.deps.alpha$include_coord_QMARK_.invokeStatic(alpha.clj:113)
at clojure.tools.deps.alpha$include_coord_QMARK_.invoke(alpha.clj:106)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:180)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:165)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:228)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:210)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invokeStatic(make_classpath.clj:59)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invoke(make_classpath.clj:52)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:70)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
#2018-10-18 12:11:09 bronsa here's a thread dump as clj hangs #2018-10-18 12:12:02 bronsa oh that ticket hasn't been resolved yet @mpenet #2018-10-18 12:12:09 bronsa so maybe it's exactly that #2018-10-18 12:19:25 mpenet Patched was applied tho i think #2018-10-18 12:20:15 pauld I'm using .397 #2018-10-18 12:20:27 bronsa it's been reverted @mpenet #2018-10-18 12:21:02 mpenet Ah #2018-10-18 12:25:27 alexmiller It was accidentally applied - I am working on alternate patch #2018-10-18 13:07:41 alexmiller @pauld can you drop a comment on TDEPS-79 with your deps.edn so I can double-check it is the same thing? #2018-10-18 13:09:27 pauld sure #2018-10-18 13:19:18 pauld @alexmiller done #2018-10-18 13:20:06 alexmiller thx #2018-10-18 16:12:21 hlolli part java question, part tools.deps question. I'm trying to leverage the git/url dep functionality on this java repo
com.kunstmusik/CsoundJNA {:git/url ""
:sha "a717dc3af6bf268b2962cac854d3fa9c0e2901a9"}
it's downloaded to ~/.gitlibs and is added onto the classpath, all good there.
But the importing is the tricky part, knowing little about java, could it be that the pom.xml file is ignored or is missing some information, https://github.com/kunstmusik/CsoundJNA/blob/master/pom.xml or is my import on the java class wrong?
(:import [com.kunstmusik.csoundjna CsoundJNA])
in my deps.edn file I have this as well
org.clojure/tools.deps.alpha {:mvn/version "0.5.460"}
#2018-10-18 16:18:02 alexmiller you can’t use a Java dep like this as something has to compile the java file to a class file. For that, you either need a build tool or an artifact with compiled classes in it. #2018-10-18 16:18:32 alexmiller git deps are only going to be useful with Clojure source only (which does not require a build) #2018-10-18 16:19:38 hlolli ah, I understand now. Thanks @alexmiller! I'll ask my friend to just publish it to maven, probably easier 🙂 #2018-10-18 18:01:14 basti Is there a way to check for the latest version of dependencies defined in deps.edn? #2018-10-18 18:01:51 dominicm Tool named depot by olical @basti #2018-10-18 18:08:41 basti Thanks I’ll check it out! #2018-10-18 19:32:03 seancorfield Why does (reader/clojure-env) care about actually building a classpath? #2018-10-18 19:35:01 seancorfield Background: we use an env var for CLJ_CONFIG so we have a known "baseline" for aliases and override-deps etc; but our actual deps require an alias -- -Adefaults -- to resolve since they all have {} for the coords and the actual versions are provided by the alias and :override-deps. So we can run CLJ_CONFIG=../versions clojure -Adefaults -Sdescribe
but if we omit the alias, (reader/clojure-env) blows up with Error building classpath. Coordinate type not loaded for library selmer/selmer in coordinate {}
Because of course we can't pass the alias into the clojure-env shell call to clojure -Sdescribe #2018-10-18 19:36:06 seancorfield @alexmiller Is there a reason that describe needs to build the classpath? If so, would you be open to a patch to allow additional arguments to be passed into clojure-env to support required aliases etc? #2018-10-18 19:39:17 dominicm I think I was supposed to open a ticket for that, sorry #2018-10-18 19:43:18 seancorfield Looks like it fails because it thinks it is "stale" and therefore runs the classpath generation code. #2018-10-18 19:55:32 alexmiller @seancorfield there is a ticket for this already (and no, I don’t think so) #2018-10-18 19:58:15 seancorfield Yup, confirmed. I modified my local clojure script to set stale=true when describe is true, and also set cp= to avoid loading the (possibly non-existent) .cpcache file, and then clojure-env runs as expected in my environment. #2018-10-18 21:21:24 seancorfield I attached a patch to that ticket matching what I changed locally to make that work @alexmiller /cc @U09LZR36F #2018-10-18 21:21:37 alexmiller thx! #2018-10-19 16:46:42 seancorfield @alexmiller Do you think TDEPS-82 will get fixed and released "soon"? We have an "outdated dependencies" step in our build chain that we'd like to run other places that just my dev machine so we'd need an updated clojure release for that. #2018-10-19 18:13:00 alexmiller I would like to fix TDEPS-79 and would then be happy to pull 82 along with that into a release #2018-10-18 19:56:31 alexmiller thx, I just modified that yesterday, will fix #2018-10-18 19:58:43 ro6 Looks like maybe the code block was supposed to end after <servers>
...
<server>
<id>my-private-repo</id>
<username>AWS_ACCESS_KEY_HERE</username>
<password>AWS_SECRET_ACCESS_KEY_HERE</password>
</server>
...
</servers>
but didn't. #2018-10-18 19:59:39 alexmiller yeah, just fixed #2018-10-19 14:31:57 jeff.terrell Is there a clj/clojure CLI equivalent to lein deps :tree? #2018-10-19 14:55:40 seancorfield The closest is clj -Stree but it doesn't show conflicts, just the resolved dependencies. Folks have asked for this a few times. You may be able to get more information with -Sverbose -Sforce. #2018-10-19 14:56:58 jeff.terrell Thanks. Nice to know for sure that this feature doesn't exist (yet). And -Sverbose -Sforce doesn't illuminate the conflict. #2018-10-19 14:58:44 seancorfield I'm not at my computer right now but there is a way to get tools.deps to tell you what versions it finds and what it selects -- which is close. #2018-10-19 14:34:07 jeff.terrell Or more generally: how can I diagnose a dependency conflict in a deps.edn project? I searched in various places (web, FAQ, deps & CLI guide and reference) but didn't see an answer. Seems like the kind of relatively undocumented thing that might go in a FAQ entry, which I'm willing to write if there's a good answer. #2018-10-19 14:38:53 jeff.terrell And by the way, is there a canonical adjective to use for projects that manage dependencies with deps.edn? I'm looking for something parallel to 'A Leiningen project' and 'A Boot project'. A 'tools.dep' project? A 'deps.edn' project? A 'clojure deps' project? #2018-10-19 14:57:10 seancorfield I've been calling them 'A clj project' to parallel the use of the tool to run them but there's nothing official as far as I know. #2018-10-19 14:58:04 jeff.terrell That's better than circumlocution, and maybe not too ambiguous. simple_smile Thanks. #2018-10-19 15:07:04 alexmiller I would say not tools.deps, but either deps.edn or Clojure tools project #2018-10-19 23:55:35 jimmy Is there a way to run clj and just have it pull down dependencies listed in deps.edn? Not start a repl or run anything, just download the dependencies. #2018-10-20 00:00:27 lilactown I run clj -Stree to do that #2018-10-20 00:29:01 alexmiller -Spath #2018-10-20 05:36:55 richiardiandrea Silly question: when clojure emits a Downloading: I guess it did not find the dep already in .m2 right?
I am trying to figure out why I have some many "Downloading" in my CI build #2018-10-20 05:42:49 richiardiandrea Yeah checking the code it seems so. It seems to me very odd that I am downloading the same deps over and over. There must be some maven setting wrong #2018-10-20 05:43:41 richiardiandrea Running on Azure Pipelines I wonder if somebody has had any problem #2018-10-20 12:21:36 alexmiller In CI, often you don’t get the .m2 dir preserved. Some providers let you mark that in different ways. #2018-10-20 15:23:10 richiardiandrea It is odd because I am running it all the tests/commands in the same shell #2018-10-22 18:39:20 jeff.terrell If you have a deps.edn project, how do you push it to Clojars?
The Clojars "pushing" wiki page, for example, gives instructions for Leiningen, Boot, and Maven, but not for deps.edn:
https://github.com/clojars/clojars-web/wiki/Pushing #2018-10-22 18:40:21 seancorfield You don't 🙂 #2018-10-22 18:40:45 seancorfield A deps.edn project can be depended on directly on GitHub. #2018-10-22 18:41:59 seancorfield A more helpful answer to your question is that you will need to use something like cambada to create a "thin" JAR and then do something appropriate to generate pom.xml and deploy it up to Clojars (Maven, I guess?). #2018-10-22 18:59:32 jeff.terrell Thanks, Sean. Yeah, I knew that about depending directly on GitHub commits. simple_smile But since the CLI tools don't work on Windows yet, I'm using Leiningen on my project, and Leiningen can't depend on GitHub coordinates (even with the lein-tools-deps plugin). #2018-10-22 19:00:58 jeff.terrell (I don't use Windows, but some members of my team do, and installing a Linux environment is a bit of a tall ask at this point.) #2018-10-22 19:09:40 seancorfield I'm surprised lein-tools-deps can't depend on git coords... boot-tools-deps supports those... #2018-10-22 19:13:41 jeff.terrell It can depend on git coords, but that functionality doesn't work on Windows. It complains about not being able to find the clojure command. #2018-10-22 19:24:54 alexmiller you can use clj -Spom to create a pom, which could be modified into a deployment path too #2018-10-22 19:25:15 alexmiller (in any answer to prior questions) #2018-10-22 19:25:46 alexmiller -Spom syncs, so if you change it the pom, those changes (outside the dependencies) will be respected #2018-10-22 19:26:01 jeff.terrell Nice, good to know, thanks Alex. #2018-10-23 17:37:38 dominicm I might have a try tonight at getting pack to work with clojars (ish). The parts are there. #2018-10-23 18:15:36 dominicm How desperate are people to avoid mvn the cli tool? I think pack already works with mvn #2018-10-23 18:37:10 seancorfield mvn just has such a nasty CLI compared to, say, lein deploy or boot deploy... #2018-10-23 18:42:07 dominicm I find it a little weird that deploy-file doesn't look at pom.xml to figure out the url automatically, but meh #2018-10-23 18:44:34 dominicm I'll write up the process with mvn and put it in pack's readme, and point at the working example. #2018-10-23 19:20:12 seancorfield To save me the trouble of actually trying to build a skinny JAR with pack and opening it up to look inside, does it add a manifest and, if so, what attributes are in it? #2018-10-23 19:27:16 dominicm The mvn deploy:deploy-file can take a pom.xml externally to use. I guess it uploads separately. You could put it inside the jar by adding a mkdir extra && cp pom.xml extra/ then passing -e extra to pack when building. #2018-10-23 19:27:39 dominicm I don't think things generally depend on the pom.xml being inside the jar though. #2018-10-23 19:37:26 seancorfield So... no manifest added nor needed? #2018-10-23 19:38:02 seancorfield And what is the difference between a "regular" skinny jar from pack and one that also has --no-libs? #2018-10-23 19:38:08 dominicm @U04V70XH6 not inside the jar, nope. You do need a pom.xml, but it's generated out of band. I didn't actually check if it's in my jar, one sec, I will #2018-10-23 19:38:36 dominicm No pom.xml in my jar. It runs fine. #2018-10-23 19:38:57 seancorfield I'm thinking back to the discussion/complaints about depstar not adding even a basic manifest into the uberjar (so you couldn't just java -jar it). #2018-10-23 19:39:41 dominicm @U04V70XH6 That's a MANIFEST.MF, you can use the -e flag in pack to add it to your classpath at build time 🙂 #2018-10-23 19:39:53 seancorfield It wasn't a big deal to add a minimal manifest with Main-Class but of course you don't actually need to -- you can just do java -cp ... clojure.main ... and run the JAR anyway. #2018-10-23 19:40:52 dominicm @U04V70XH6 regarding --no-libs, the general idea of skinny jar mode is to produce something that's really cheap to rsync or something onto s3. So you generally output:
app.jar
lib/clojure-1.8.0.jar
lib/github-0.1.0.jar
And then upload app.jar + any files from libs that are missing.
This was requested by a company doing java, this is their upload process. #2018-10-23 19:40:58 seancorfield Yeah, my point/question was "Does pack add a manifest by default?" and the answer seems to be "no" and, implied, does a library published to Clojars need a manifest and I think the answer is also "no". #2018-10-23 19:41:49 dominicm Capsule/Onejar do add a manifest, because it's required for them to function. I'm not entirely happy with the process though. I think more could be done here, but there's nothing to drive change currently. #2018-10-23 19:41:51 seancorfield Ah, so "skinny" means no libs in the JAR but instead copy them to a lib/ folder? #2018-10-23 19:42:31 dominicm @U04V70XH6 Yep! #2018-10-23 19:43:34 dominicm In the future I might add a dedicated maven namespace to pack. I want to end up where you go via use-case rather than "underlying details". UberJAR complicates this a little, as there's choice. #2018-10-23 19:44:30 seancorfield Part of me is just contemplating what I might do with my fork of depstar -- to keep it as minimal as possible but open up the possibility of producing a thin JAR and being part of a workflow that could produce a pom.xml and mvn it up to Clojars... or another repo... #2018-10-23 20:23:24 dottedmag Pardon my ignorance, does Lein use mvn to upload? #2018-10-23 20:24:28 andy.fingerhut Probably more likely to find someone who knows the answer to that in #leiningen channel #2018-10-23 20:26:03 dottedmag Figured it out already 🙂 It uses cemerick/pomegranate - a wrapper around Eclipse Aether. #2018-10-23 20:43:53 arrdem tools.release(.alpha) 😉 #2018-10-24 03:18:28 seancorfield I gave in and put seancorfield/clj-new and seancorfield/depstar on Clojars. Releases 0.5.4 and 0.1.2 respectively. JARs built with (my fork of) depstar. Initial pom.xml built by clj -Spom. Pushed to Clojars with mvn deploy:deploy-file (ugh!). #2018-10-24 17:18:59 dominicm Topic of today. I want multiple aliases which compose and start multiple things. The goal would be using able to start a program like normal, but perhaps side load nrepl in.
Best idea I have so far is a custom main and jvm options. The custom main would evaluate code based on the provided options, and then finally run the -m it was provided.
Any other suggestions for ways to interface with such a thing?
I did consider taking a list of mains, but then you have to detect when they are done in an arbitrary way, but that might be fine by looking at the thread state? Also main-opts don't concatenate. #2018-10-25 15:20:11 xfyre Interesting, let me check this in Xcode. I was experiencing reload issues before I even tried to upgrade Figwheel #2018-10-25 15:34:51 xfyre I’m seeing
2018-10-25 18:34:41.958455+0300 DcnetApp[90232:23128804] [] nw_socket_handle_socket_event [C224.1:1] Socket SO_ERROR [61: Connection refused]
#2018-10-24 18:55:52 seancorfield @dominicm I thought about this long and hard as we switched our dev/test/build pipeline over to clj from boot -- and ended up with the "custom main and jvm options" approach, with several small "microprojects" containing the various -main we want that are added via aliases... :main-opts don't concatenate for a (good) reason but that does sometimes make it hard to "compose" multiple tasks in a "build pipeline". Our compromise there was to create a build shell script that runs clojure for each set of command line arguments (loosely... it implements a sort of mini-DSL around aliases and subprojects to make it easy for us to kick off a sequence of tasks from a single command line). #2018-10-24 18:56:55 seancorfield (as of yesterday, we no longer use boot for any of our dev/test/qa/prod stuff -- with two legacy exceptions, one of which I'm working on right now 🙂 ) #2018-10-24 19:28:02 dominicm @seancorfield Just so I am clear, you have a custom main which calls other -main? Or do you look at JVM opts in order to determine what to start in your one custom main? #2018-10-24 19:29:53 seancorfield "It Depends". We have some -main functions that use the supplied command line arguments to decide what to run (namespace-qualified function names, usually). But also, now we have our little build wrapper shell script, we often just run clojure multiple times with different (main) options... #2018-10-24 19:30:24 seancorfield But the former approach could easily use JVM options instead of command line arguments. Or as well as. #2018-10-24 19:31:20 seancorfield (the main issue is determining what arguments -- command line or otherwise -- to call each of the functions with, based on just a single -main entry point and its own command line arguments) #2018-10-24 19:32:03 arrdem it’d be interesting to have a “multi clojure.main” that took say an EDN array of mains and ran each one in its own future / send-off. #2018-10-24 19:32:22 arrdem or let you have “mixins” that get sent off and one “main”. #2018-10-24 19:33:12 dominicm @arrdem I'm toying with that idea. The hard limitation being that if you actually want the main to run once and then block, it's more difficult. Also, if you want ordering (e.g. producing some state picked up for the next main), then you have to "detect" when the main is done. #2018-10-24 19:33:52 arrdem Concatenating programs into a single JVM seems like … a job for a real program not some crazy main. We have a thing which does that, it’s a REPL 😉 #2018-10-24 19:34:55 arrdem For Katamari I’ve been toying with the idea of how to let kat reuse a JVM between taks if your dependencies haven’t changed. Not sure you can actually blow away enough Clojure state (in an unmodified Clojure) to do that safely or if it buys you anything. #2018-10-24 19:35:23 dominicm I'm trying to leverage existing standards (e.g. -main). But I agree. I'm feeling like I might have to bite the bullet and "force" something new onto people for daemons like nrepl. #2018-10-24 19:36:07 arrdem Yeah. Remote daemonized nREPL servers with a client hooked to your terminal is the best I’ve got for stuff like this. #2018-10-24 19:36:54 arrdem I need to take a poke at pack.alpha, looks like it may provide more of the machinery I want for Katamari’s jar, uberjar and maybe pom targets than depstar does. #2018-10-24 19:38:02 dominicm @arrdem I'm trying to move towards having some kind of API for consumers. But I've prioritized -main support for now. I'm very welcome to improvements to the "api" 🙂 #2018-10-24 19:38:37 arrdem @dominicm cool - I’ll see what I find 😉 #2018-10-24 19:38:54 arrdem kat is already a pile of alpha APIs what’s one more. #2018-10-24 19:39:43 dominicm Hehe. #2018-10-24 19:39:52 arrdem @seancorfield curious what wound up in your build script. #2018-10-24 19:44:43 seancorfield @arrdem Well, our basic command to run anything is essentially cd <repo>/clojure/<subproject>
CLJ_CONFIG=../versions clojure -A:defaults:<alias> <some> <args>
so mostly the build script maps pairs of <task> <subproject> on the command line into runs of that command pair. #2018-10-24 19:45:58 seancorfield Since arguments are inherently variadic, we treat <task> <subproject> as a command with no args and [ <task> <subproject> <some> <args> ] as a command with some args. So build parses the [ and ] delimiters around commands. #2018-10-24 19:46:27 arrdem Interesting. #2018-10-24 19:47:45 seancorfield Plus a couple of convenient conventions: <task>+test means "map <task> to its alias(es) and also add :test:test-deps; and <task>s means "run <task> for each of the (subproject) arguments provided" -- so you can easily build multiple uberjars or run multiple subprojects' tests. #2018-10-24 19:48:38 seancorfield Oh, and <task>-all which expands a list of subprojects to include all the others (recursively) that are pulled in as :local/root deps (so we can easily "test a subproject and everything it depends on") #2018-10-24 19:49:20 seancorfield Very minimal really. And of course we could do it all from the command line but... we're lazy programmers... #2018-10-24 19:50:31 seancorfield Everything beyond that "DSL" is handled with aliases (and command line arguments passed to clojure). #2018-10-24 19:52:03 arrdem Hum. Do you just have standardized eg /test paths for the -all alias or how did you get dependees tests recursively included. #2018-10-24 19:54:04 seancorfield A monorepo with subprojects all side-by-side in the clojure directory, each with src and test folders beneath them. Is that what you meant? #2018-10-24 19:54:21 arrdem Yep. #2018-10-24 19:54:26 arrdem Cool, thanks for the info! #2018-10-24 19:54:36 arrdem Interesting to see what other folks are coming up with. #2018-10-24 19:55:09 seancorfield We deleted most of our 2,000 line build.boot file yesterday 🙂 #2018-10-24 19:55:28 seancorfield I expect to delete almost all of the rest of it by week's end. #2018-10-24 20:14:33 dottedmag @seancorfield How big is the replacement? #2018-10-24 20:16:39 seancorfield A very well commented and spaced out bash script with about 70 lines of help output -- total just under 300 lines 🙂 #2018-10-24 20:17:09 dottedmag So, 2000 lines of build.boot was just a dispatcher? #2018-10-24 20:17:24 seancorfield No... it's kinda hard to explain 🙂 #2018-10-24 20:18:26 seancorfield We used to dynamically determine cross-subproject dependencies using tools.namespace -- seeing which namespaces a subproject required and which subprojects those were actually defined in. All that is gone and replaced with explicit :local/root dependencies now. #2018-10-24 20:20:20 seancorfield Plus we had accrued a lot of now-unused build tooling as we evolved the project over several years (we switched to Boot at the end of 2015). We had a lot of per-subproject-specific stuff in the Boot file which was never made generic (which the shell script does). #2018-10-24 20:21:36 dottedmag I see. #2018-10-24 20:23:28 seancorfield Boot had encouraged us to build complex tooling because it was easy to do so. The reason we switched from Leiningen to Boot three years ago was because we wanted to do more programmatic stuff -- which was much, much easier in Boot -- but we sort of got carried away 🙂 And moving to clj and deps.edn and aliases kind of focuses the mind on what's actually required. #2018-10-24 20:31:53 dottedmag It reminds me of one of build steps (implemented in Go, to boot) I'm guilty of writing: it extracts from a Go binary a list of versions of packages it was built from, parses list of packages current module is dependent upon, and then decides whether that binary needs to be rebuilt. And to add to the insult, the list of binaries to check is created by parsing a Go source file. #2018-10-24 20:33:00 dottedmag Very nifty, cuts down maybe 10s from the full build, but it's more than 300 lines of code just for that. #2018-10-24 20:34:55 seancorfield Ah, yes, an easy trap to fall into 🙂 #2018-10-24 20:36:38 arrdem Curious what makes you see that as a trap. #2018-10-24 20:37:51 seancorfield When your build is "just code" it's easy to just start writing code -- without thinking about whether it is a) needed or b) an over-complicated way to achieve your goal. #2018-10-24 20:51:05 dottedmag Also this code is usually easy and fun to write, so one can waste enormous amount of time on it. #2018-10-25 05:09:39 steveb8n I’m trying to use a private bitbucket repo as a git dep but getting “auth fail” from the jgit layer. I’ve tested ssh access from the terminal and that’s working. anyone else had any luck with this? #2018-10-25 05:11:05 seancorfield I think that is only partially supported at he moment...? #2018-10-25 05:11:19 seancorfield (I'm trying to find the public discussions about it) #2018-10-25 05:12:51 seancorfield Hmm, OK, looks like ssh keypair should work for git deps... #2018-10-25 05:13:45 seancorfield In particular "Note: user/password authentication is not supported for any protocol." #2018-10-25 05:14:10 seancorfield How do you have your repo configured in deps.edn @steveb8n? #2018-10-25 05:15:13 steveb8n thx for quick response. I’m using an SSH key with ssh-agent on OSX i.e. not using un/pw #2018-10-25 05:15:41 steveb8n in deps.edn I follow the pattern I’m using for some public repos i.e. same keys #2018-10-25 05:15:59 steveb8n only difference is :git/url is ssh: instead of https: #2018-10-25 05:16:37 steveb8n when I run ssh -v in CLI, that shows successful connection #2018-10-25 05:19:08 steveb8n hmm, it looks like I am missing a “registered identity” #2018-10-25 05:19:14 steveb8n I’ll dig deeper into that #2018-10-25 05:19:45 steveb8n and will report back #2018-10-25 05:30:02 steveb8n I found the problem. I am using the OSX ssh-agent with the OSX keychain and it’s not persistent between reboots #2018-10-25 05:30:12 steveb8n ssh-add -K fixed it #2018-10-25 05:30:24 steveb8n thanks for pointing me in the right direction #2018-10-25 05:33:30 seancorfield Cool! The whole ssh keys/identity thing can be a bear... #2018-10-25 05:44:33 steveb8n yep. in this case it’s a different behaviour in zsh CLI vs jgit (part of tools.deps). I’ve written it down in my readme for next time 🙂 #2018-10-25 10:47:13 steveb8n @seancorfield a heads up for future. when using git private deps in CI, you must invoke the ssh-agent before invoking clojure i.e. eval #2018-10-25 10:48:22 steveb8n those double backticks should be single (I’m no good at slack code #2018-10-25 10:48:33 steveb8n this should probably go in some docs somewhere #2018-10-25 08:14:31 Christian Johansen Whenever I have compilation errors in my code, starting a REPL with clojure -A:dev only gives me this: #2018-10-25 08:14:34 Christian Johansen clj -A:dev
2018-10-25 10:06:47.433:INFO::main: Logging initialized @10380ms
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)
#2018-10-25 08:14:39 Christian Johansen can this be improved somehow? #2018-10-25 08:14:56 Christian Johansen a stack trace, anything to point me towards the problem? #2018-10-25 08:19:42 dominicm @christian767 my immediate thought is to try Clojure 1.10 alphas. As there's been some big changes around this stuff. #2018-10-25 08:20:41 Christian Johansen this is with 1.10.0-alpha8 #2018-10-25 08:20:58 Christian Johansen I’ll try an even newer one #2018-10-25 08:30:57 Christian Johansen with clojure-tools-1.9.0.397 and Clojure 1.10.0-beta4: #2018-10-25 08:31:00 Christian Johansen clj -A:dev
2018-10-25 10:30:49.453:INFO::main: Logging initialized @11217ms
Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)
#2018-10-25 08:31:02 Christian Johansen 😢 #2018-10-25 08:32:05 seancorfield So that's failing before it even gets Clojure up and running... #2018-10-25 08:32:39 seancorfield You'll need to share some of your deps.edn (possibly both your ~/.clojure version and your project one). #2018-10-25 08:33:05 seancorfield Does it start a REPL without -A:dev? #2018-10-25 08:33:15 seancorfield If so, your :dev alias is the problem. #2018-10-25 08:33:20 Christian Johansen good tip #2018-10-25 08:33:26 Christian Johansen sure does #2018-10-25 08:33:52 seancorfield OK, so what's in your :dev alias? #2018-10-25 08:33:58 Christian Johansen found the culprit #2018-10-25 08:34:17 Christian Johansen I have dev/user.clj, and in it I’m still using a dependency I ditched yesterday #2018-10-25 08:34:26 Christian Johansen [yonatane.timbre-json :as timbre-json] #2018-10-25 08:35:07 Christian Johansen is there something I could’ve done to have this reported in an understandable way? #2018-10-25 08:35:42 Christian Johansen (there was an :extra-paths in :dev) #2018-10-25 08:35:44 seancorfield Maybe wrap the contents of your user.clj file in (try ... (catch Throwable t (println "dev/user.clj broke" t))) #2018-10-25 08:36:23 Christian Johansen can you try around the ns form? #2018-10-25 08:36:36 Christian Johansen anyway, what if the error was in my source? can’t try-catch all my code #2018-10-25 08:36:43 seancorfield Sure. ns is just code. #2018-10-25 08:37:06 Christian Johansen seems strange to have to try-catch all my code to see compilation failures #2018-10-25 08:37:33 seancorfield Well, true, if your user.clj won't actually compile that's just going to break. user.clj is pretty special so you really should be very sparing about what you do in it. #2018-10-25 08:37:52 seancorfield You could always debug it with clj dev/user.clj I think? #2018-10-25 08:38:06 Christian Johansen interesting #2018-10-25 08:38:26 seancorfield (without the -A:dev alias, it wouldn't be on your classpath so it wouldn't automatically run at Clojure startup) #2018-10-25 08:38:48 Christian Johansen my impression was that I always got this behavior with any compilation errors, but you’re saying this is happening because it’s in user? #2018-10-25 08:39:10 seancorfield Right, because it's executed as part of starting Clojure. #2018-10-25 08:39:24 Christian Johansen I see #2018-10-25 08:39:41 Christian Johansen by extension, this would happen with anything required from user too then? #2018-10-25 08:40:07 seancorfield Well, wrapping the ns form in try/`catch` should deal with that. #2018-10-25 08:40:14 Christian Johansen right #2018-10-25 08:40:39 Christian Johansen leiningen manages to report this more clearly, I wonder why that is #2018-10-25 08:40:58 seancorfield Because Leiningen starts two JVMs perhaps? #2018-10-25 08:41:03 Christian Johansen aha #2018-10-25 08:41:40 seancorfield OK, it's crazy late hear. I'm off to bed (nearly 2am). #2018-10-25 08:41:54 Christian Johansen whoa, It’s 10 in the morning over here 🙂 #2018-10-25 08:41:57 Christian Johansen thanks for your help #2018-10-25 08:41:58 seancorfield Hopefully other folks will be coming online soon if you have other Qs. #2018-10-25 08:42:04 Christian Johansen :+1: #2018-10-25 09:16:56 Christian Johansen it’s quite the buzzkill for new tools.deps users #2018-10-25 10:40:50 Christian Johansen not putting code in user.clj seems like the better approach for now. is there a tools.deps analog to leiningens :repl-options {:init-ns my-app.dev}? #2018-10-25 10:43:21 dominicm In edge we solve this by having a dev.clj #2018-10-25 10:50:35 Christian Johansen how do you load it? #2018-10-25 12:24:29 alexmiller I suspect this is really a Clojure problem, not a tools.deps problem #2018-10-25 12:26:05 Christian Johansen yes, after discussing with more people I realize it probably is #2018-10-25 12:26:21 Christian Johansen sorry for misplacing the Jira issue… #2018-10-25 12:46:00 alexmiller No worries, it can be moved #2018-10-25 19:29:53 arrdem thanks for the depstar fix @seancorfield - it works like a charm! #2018-10-25 19:32:22 seancorfield I plan to actively maintain my fork, going forward, because we're relying heavily on it at work. #2018-10-25 19:33:56 seancorfield I've also started using it for packaging library JARs for deployment to Clojars now (there's an hf.depstar.jar entry point for thin JARs now, then clj -Spom -- and one-off updates to add/update license, SCM, etc -- and then mvn deploy:deploy-file). #2018-10-25 20:33:46 arrdem Cool. I’ll probably be looking long and hard at depstar vs pack.alpha shortly just because I need to start generating manifests from Katamari, but if I stick with depstar I’d be happy to support your copy if its intent is to be active. #2018-10-25 20:46:55 ghadi My experiment worked! #2018-10-25 20:47:45 ghadi The experiment was: Git deps allow you to be a shitty maintainer because someone can just fork and change the url, without having to deploy an artifact #2018-10-25 22:52:08 zane I've been waiting for that PR to be merged. 😞 #2018-10-25 20:49:32 ghadi @seancorfield didn't have to do anything to unblock himself on depstar, which I neglected to maintain #2018-10-25 20:50:08 arrdem 🤔 maybe {:mvn/version } could let you pass it an override lib to get the same place without refactoring your deps lists … #2018-10-26 05:30:30 dominicm There's a ticket for this #2018-10-25 21:03:18 seancorfield :rolling_on_the_floor_laughing: @ghadi Although I have now deployed artifacts of seancorfield/depstar -- using the modified version of depstar! 🙂 #2018-10-25 21:04:45 ghadi if you break the API in a significant way I'll kindly ask you to rename #2018-10-25 21:04:54 ghadi but I'm happy that you're happy sean! #2018-10-25 21:05:25 ghadi It's a really good name if I do say so -- it's tar for deps, and a deathstar #2018-10-25 21:07:27 seancorfield I like the simplicity of it. So far, I've only fixed bugs in the existing code and added a new namespace/entry point (for thin JARs). No API changes 🙂 #2018-10-25 22:52:17 steveb8n when I use a :local/root dep, I’m finding that all my aliases are unavailable. workaround is to add copies of all aliases (with empty maps) into the dependencies deps.edn. Is this known? #2018-10-25 23:22:48 arrdem @steveb8n AFAIK this is intended behavior - clj/`clojure` do not yet support merging “shared” extra deps files eg. for aliases besides the install global file and the user config. #2018-10-25 23:23:50 arrdem I think seancorfield has done some hacking to be able to share a defaults file within a monorepo as have I. #2018-10-25 23:25:07 steveb8n hmm, in that case 1/ it would be good to doc this in the :local/root docs and 2/ does the workaround I discovered make sense? #2018-10-25 23:26:39 steveb8n I’m good to go with the workaround but just asking for others who will inevitably hit this #2018-10-25 23:30:36 steveb8n that’s my version of “asking for a friend” #2018-10-25 23:42:30 seancorfield It doesn't make sense for aliases in a deps project you depend on to bleed into your project's deps. #2018-10-25 23:43:17 seancorfield :local/root is nothing special -- just a want to specify a dependency. You wouldn't want all the aliases from a random :git/url deps file to bleed into your project, would you? #2018-10-25 23:44:18 seancorfield The aliases available to you should always be: system deps + user deps (which can be anywhere based on CLJ_CONFIG env var) + project deps. #2018-10-25 23:58:49 arrdem I’ve been kicking around whether it makes sense to want to build deps with aliases - eg what if I wanted to let every one of my dependencies apply its :test alias so that I get a single classpath with all the machinery required to run all my deps’ tests. #2018-10-26 00:00:15 arrdem Kinda gives you a way to have a parameterized debug build or such. #2018-10-26 00:04:23 steveb8n interesting idea. I guess we’ll see some best practices emerge around primary and dependent aliases #2018-10-26 07:09:49 olivergeorge This is probably a silly question. Should it be possible to clone clojure locally and use it via deps.edn via :local/root? #2018-10-26 07:24:08 dominicm Nope! It has Java in #2018-10-26 07:26:07 olivergeorge Thanks. Not really surprised. I did get it working by building first (e.g. ant install) and pointing at the jar with {:deps {org.clojure/clojure {:local/root "/Users/olivergeorge/repos/fork-clj/clojure/target/clojure-1.10.0-master-SNAPSHOT.jar"}}} #2018-10-26 07:26:08 olivergeorge Was wondering more broadly how easy it was getting to tweak clojure source and see it in a REPL. #2018-10-26 12:40:52 alexmiller I do something similar when working on Clojure #2018-10-26 21:49:51 olivergeorge Cool. Pretty productive given the clojure build process is all of 10s on a 4yo macbook #2018-10-27 06:44:00 dominicm I've got a complete first pass of "ednup" a tool for rewriting edn without losing whitespace. It's not great, but it is definitely sufficient to handle resolve tags! Don't expect this to work beyond this commit, I'm going to clean up with new namespaces, but:
{:paths ["src"]
;; cool comment
:deps
{potemkin {:mvn/version "0.1"}
tools.deps.alpha {:git/url ""
;; totally rad
:tag "tools.deps.alpha-0.5.460"}
;; woah dude
tools.deps.alphax {:git/url ""
:tag "tools.deps.alpha-0.5.460"
;; zoom
:sha nil}}}
Running
~/src/github.com/SevereOverfl0w/clj-resolve-tags master* 13s
❯ clj -Sdeps '{:deps {resolve-tags {:git/url "" :sha "44315f501e3b7fba40961563ba7fcfd0441225fa"}}}' -m resolve-tags not-deps.edn
Results in
{:paths ["src"]
;; cool comment
:deps
{potemkin {:mvn/version "0.1"}
tools.deps.alpha {:git/url ""
;; totally rad
:tag "tools.deps.alpha-0.5.460"
:sha "ca2aa1eb320687ce940a4f142c8d1af8f9a018bc"}
;; woah dude
tools.deps.alphax {:git/url ""
:tag "tools.deps.alpha-0.5.460"
;; zoom
:sha "ca2aa1eb320687ce940a4f142c8d1af8f9a018bc"}}}
#2018-10-27 19:12:16 andy.fingerhut Is there already an issue suggesting the addition of a "show version of clj/clojure and exit" option to clj and/or clojure? If not, would one be welcome? Seems useful in some debug interactions between people. #2018-10-27 19:13:36 dominicm It does I think? -Sdescribe #2018-10-27 19:16:16 andy.fingerhut Got it. thx. Didn't realize that, and didn't see the more traditional -v or --version command line options. #2018-10-27 19:17:14 dominicm Have those been added? Neat. I haven't tracked really closely. #2018-10-27 19:17:29 andy.fingerhut I don't think so. I didn't see them. #2018-10-27 19:22:54 dominicm Ah. I will stick to describe them 😊 #2018-10-27 23:43:16 jeff.terrell I've also been surprised by the lack of -v / --version options, FWIW. #2018-10-28 05:10:51 richiardiandrea I can trigger a (swap! verbose-surprise-count inc) as well :smiling_face_with_smiling_eyes_and_hand_covering_mouth: #2018-10-29 04:00:23 steveb8n When using :git/url deps, is there any way to emulate latest or snapshot style behaviour? I'm having to manually updates shas a lot. #2018-10-29 06:36:53 dominicm I don't think so - as an intentional decision. #2018-10-29 06:37:12 dominicm You could write a script to update them automatically though. #2018-10-29 08:13:33 steveb8n Interesting idea. It seems like I won't be the only one wanting this. I wonder if there's another best/good practice to emulate maven in this way #2018-10-29 10:13:30 dominicm The code is about 100 lines. You could modify the resolve-tags library I write quite easily. #2018-10-29 10:33:38 steveb8n Hmm, maybe I should be writing a new “procurer”. I’ll look into that as a solution. thanks for the nudge in a new direction #2018-10-29 12:08:35 alexmiller It does, but tools.deps does not (gitlibs is intentionally broader and more generic) #2018-10-29 09:36:31 arrdem If you're coordinating changes between many repos all the time I think it's worth considering whether the apparent decoupling you have at repo boundaries maps at all to stable designed API boundaries or whether you'd be better off admitting that you don't yet have decoupling and would be better served putting everything in one place for now. #2018-10-29 10:31:28 steveb8n Good point. In my case it’s not de-coupling I want, but shared specs. The specs are changing a lot currently, hence the need for regular updates #2018-10-29 09:38:59 arrdem In slightly related news I got a Katamari v0.0.5 out the door which successfully does content addressing of build products (jars, uberjars, javac'd classfiles). At this point it's fully self-bootstrapped and stable as such. Incremental rebuilds were the big thing I wanted to get working before this week's meeup - if I can get repl and test tasks sorted out I'll be happy enough to really try and push this thing.
https://github.com/arrdem/katamari/releases/tag/v0.0.5 for your amusement. #2018-10-29 10:58:34 Christian Johansen I cleared out ~/.m2/repos/org/apache, and now I’m seeing this error with tools.deps: #2018-10-29 10:58:40 Christian Johansen Exception in thread "main" org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.httpcomponents:httpcore:jar:4.4.9
#2018-10-29 10:59:00 Christian Johansen after googling a little, it seems to go away with this in deps.edn: #2018-10-29 10:59:02 Christian Johansen :mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2018-10-29 10:59:06 Christian Johansen what’s going on here? #2018-10-29 11:33:42 Christian Johansen btw, I can observe that the jar in question is both being downloaded by the Clojure process, and is available in my ~/.m2 after its run (and failed) #2018-10-29 12:10:14 alexmiller Those maven repos are already set in the install deps.edn so that shouldn’t make much difference #2018-10-29 12:10:41 alexmiller Can you post the entire deps.edn for repro? #2018-10-29 13:18:19 jerger_at_dda I had just the time to watch "Dependency Heaven" an like the arguments.
I'm working quite a lot with git dependencies on systems like gitbook (make books out of git-repos containing md) or smeagol (a git-repo / MD based wiki)
But I find it quite hard to find a good abstraction for addressing git resources.
Are there any plans to support other git servers like gitlab (the SaaS and all the on prem instances) or gitblit, gittea et al?
What about private repos?
In context of privisioning git repos I use this schema: https://github.com/DomainDrivenArchitecture/dda-git-crate/blob/master/doc/reference_domain.md
The relevant parts are Repository and GitCredential. These informations will lead to completely different urls like
#2018-10-29 14:00:16 dominicm @jerger_at_dda gitlab is supported? Anything git takes should work I think? #2018-10-29 14:00:30 dominicm Also, I think you'd be interested in tools.gitlibs 🙂 #2018-10-29 14:14:13 jerger_at_dda okay I see ... switching between https & ssh references is not a usecase for u - so your implementation not has to cope with url-magic. #2018-10-29 14:16:26 jerger_at_dda Gitlab & Github will both have a git user for ssh access (url is sth like >) #2018-10-29 14:18:19 jerger_at_dda #2018-10-29 14:18:56 dominicm #2018-10-29 14:19:19 dominicm #2018-10-29 14:29:39 jerger_at_dda #2018-10-29 14:31:15 dominicm #2018-10-29 14:35:39 jerger_at_dda #2018-10-29 14:35:53 jerger_at_dda #2018-10-29 14:39:59 jerger_at_dda #2018-10-29 14:42:05 dominicm #2018-10-29 14:43:00 jerger_at_dda #2018-10-29 14:43:35 jerger_at_dda #2018-10-29 14:46:54 jerger_at_dda #2018-10-29 14:47:18 dominicm #2018-10-29 14:47:39 jerger_at_dda #2018-10-29 18:25:08 alexmiller #2018-10-29 18:26:03 alexmiller #2018-10-29 18:26:40 alexmiller #2018-10-29 18:28:24 alexmiller #2018-10-29 18:36:21 dominicm #2018-10-29 18:37:11 dominicm #2018-10-29 19:55:19 dominicm #2018-10-29 21:08:58 dominicm #2018-10-29 21:11:01 hiredman #2018-10-29 21:11:48 hiredman #2018-10-29 21:11:59 ghadi #2018-10-29 21:16:09 dominicm #2018-10-29 21:21:38 alexmiller #2018-10-29 21:22:00 alexmiller #2018-10-29 21:23:02 alexmiller #2018-10-29 21:27:46 alexmiller #2018-10-29 21:27:59 alexmiller #2018-10-30 08:16:29 Christian Johansen @alexmiller turns out it was the pack.alpha bug you were just talking about 🙂 #2018-10-30 08:16:30 Christian Johansen thanks! #2018-10-30 08:27:23 dominicm I've bumped the injector, so running the new version of the command should auto-fix it. #2018-10-30 08:28:02 Christian Johansen just upgraded the sha by hand, worked perfectly! #2018-10-30 08:43:30 dominicm Great! 🙂 #2018-10-30 08:44:22 dominicm I do have an intention to create a "registry" at some point, which is integrated with depot or something like it for finding tools like this and keeping them up to date. #2018-10-30 12:30:16 Christian Johansen that would be nice #2018-10-30 12:30:53 Christian Johansen I don’t mind the git shas though, in fact I quite enjoy that they don’t try to sell the illusion of “semantic versioning” #2018-10-30 12:32:25 dominicm I don't like that I can't easily see progression, but otherwise I think they're fine. #2018-10-30 15:25:47 neural is lein or boot more used in projects than clj/clojure? i like deps.edn but i see that the majority of projects are using lein or boot! why is that? #2018-10-30 15:32:50 dominicm It is more used. Those tools have been around much longer than deps.edn, which is still somewhat alpha. #2018-10-30 15:50:21 alexmiller deps.edn is not alpha, but is less than a year old #2018-10-30 16:39:20 dominicm My mistake. Assumed it was under t.d.a. #2018-10-30 16:41:07 seancorfield It's reasonable to file issues with projects that don't have deps.edn, asking them to add one. #2018-10-30 16:41:24 seancorfield I've been trying to make sure I do that with every project I maintain. #2018-10-30 16:43:13 neural @seancorfield nice #2018-10-30 16:43:32 seancorfield (in fact, in the Contrib libraries I maintain, I've removed project.clj because it's misleading -- many people assume the version in that file, or the specific dependencies in that file, are definitive, whereas for Contrib projects, it's driven by pom.xml) #2018-10-30 16:51:08 neural maybe this is an other channel related question... but you guys know if clj-refactor/refactor-nrepl works with deps.edn??? #2018-10-30 17:07:12 seancorfield @neural.works.com As far as I know, the most recent version of CIDER works with clj/`deps.edn` -- but maybe better to ask in #cider for more detail...? #2018-10-30 17:08:09 neural @seancorfield tks #2018-10-30 19:43:56 dominicm It does. I set that up. #2018-10-31 21:08:46 timgilbert Say, anybody got something that works similarly to lein-try, where I can lein try clojure-lanterna and fire up a REPL with the latest version loaded? #2018-10-31 21:11:28 jeff.terrell Relative newbie here, but I think the -Sdeps option covers that use case, no? #2018-10-31 21:12:05 timgilbert I'm thinking in a bare directory with no existing deps.edn present #2018-10-31 21:12:14 ghadi yup, that's what it's for #2018-10-31 21:12:17 dominicm I would use find deps #2018-10-31 21:15:00 timgilbert Think that'll work, thanks @dominicm #2018-10-31 21:50:15 seancorfield clj -Sdeps '{:deps {clojure-lanterna {:mvn/version "RELEASE"}}}' -- or you can use "LATEST" to look for snapshots instead. #2018-10-31 21:50:50 seancorfield I use a command like that all the time when I just want to try out a library without any setup. #2018-10-31 21:54:04 seancorfield That looks like a fun library @timgilbert -- are you planning to build something public or is it just for fun? #2018-10-31 23:02:17 timgilbert Just for fun, was looking at some roguelike things from a HN post earlier today #2018-11-01 09:28:14 borkdude Is it possible to combine multiple R aliases into one alias? #2018-11-01 09:42:01 dominicm How do you mean? #2018-11-01 09:55:39 borkdude Say alias A says extra deps d1 d2, alias B says extra deps d3 d4, now I want to make an alias AB which combines A and B so it has extra deps d1 .. d4 #2018-11-01 10:07:48 dominicm @borkdude there's no inheritance. So the only way to do it is by modifying the edn file. I've been writing https://github.com/SevereOverfl0w/ednup with the goal of it eventually supporting this. #2018-11-01 16:50:03 timgilbert Say, I'm trying to figure out how to effectively deal with the .cpcache directory in circleci. The way it works is that circleci can store a directory in a cache which is then available to subsequent steps (although the subsequent steps might themselves run on different VM instances). #2018-11-01 16:50:51 timgilbert So what I currently do is run clojure -A:foo:bar -Spath to download a bunch of stuff and then cache the .m2 and .cpcache directories #2018-11-01 16:51:51 timgilbert My question is, will tools.deps create a single .cpcache entry for the combined :foo:bar aliases, or will it create a :foo cache and a :bar cache? (Or all three?) #2018-11-01 16:52:25 timgilbert I'm wondering if it would be more efficient to run clojure -A:foo -Spath; clojure -A:bar -Spath #2018-11-01 16:52:27 alexmiller combined #2018-11-01 16:52:37 alexmiller the cache key is based on the set of aliases used #2018-11-01 16:52:56 timgilbert Aha, makes sense. Thanks @alexmiller #2018-11-01 17:35:27 fmjrey Hey, I'm migrating some lein project to deps.edn and somehow it looks like my dev/user.clj isn't loaded in the repl, even though I use clj -R:dev and the :dev alias does have :extra-paths ["dev" "fixtures"]. My user.clj is very simple:
(:require
[com.stuartsierra.component.user-helpers :refer [dev go reset]]))
clj -Stree -R:dev shows the component lib is loaded.
Still it's as if user.clj isn't loaded:
$ clj -R:dev dev/user.clj
$
No output or error.
$ clj -R:dev
Clojure 1.9.0
user=> (dev)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: dev in this context, compiling:(NO_SOURCE_PATH:1:1)
user=>
What could I be missing? #2018-11-01 17:44:07 hiredman use -A not -R #2018-11-01 17:47:55 fmjrey Thanks, feel stupid now, wondering how I ended up using -R #2018-11-01 17:49:09 hiredman similar questions have come up a few times, it makes me wonder if there is a tutorial or documentation somewhere that is using -R in examples where it should be using -A #2018-11-01 17:51:57 fmjrey > You can add this dependency to your classpath by adding the :bench alias to modify the dependency resolution: clj -R:bench. #2018-11-01 17:57:17 fmjrey Not even sure I understand the difference between -A and -R from the cli doc #2018-11-01 17:51:39 richiardiandrea question about CI and .m2 - is there anything I have to do to set Clojure CLI on a machine for being able to cache artifacts in .m2? #2018-11-01 17:52:11 richiardiandrea this machine does not seem to have Maven, do I need it? or .m2/settings.xml #2018-11-01 17:52:33 hiredman nope #2018-11-01 17:53:31 hiredman if you are ci'ing inside some kind of container you might want to set things up so ~/.m2 is stored in a way that persists across different runs #2018-11-01 18:42:30 richiardiandrea is .m2 created for me? #2018-11-01 18:43:00 hiredman yes #2018-11-01 18:43:46 hiredman tools.deps will store git deps in ~/.gitlibs too #2018-11-01 18:46:08 richiardiandrea ok thanks, my CI executes things in a weird way I think...every call to clojure resets .m2 so it downloads everything again, investigating... #2018-11-01 18:48:00 hiredman yeah, you are likely building in some kind of container environment with nothing persisted between builds #2018-11-01 18:48:03 richiardiandrea I am on Azure Pipeline.. #2018-11-01 18:48:08 richiardiandrea yeah.. #2018-11-01 18:50:01 hiredman in theory that should be fine, m2 is a cache, so if the cache is missing it will just rebuild it, but it is much better to persist m2 #2018-11-01 18:50:14 hiredman you actually might be running in to another caching issue too #2018-11-01 18:50:38 richiardiandrea yeah I see it is redownloading the whole thing #2018-11-01 18:51:22 hiredman the clj tool fetches deps and builds the classpath once and sort of caches that, if the deps it fetches go missing, but the cache file it generates is still there, it will blow up #2018-11-01 18:51:55 richiardiandrea makes sense yeah #2018-11-01 18:51:59 richiardiandrea thank you so much #2018-11-01 19:45:18 richiardiandrea which command creates .m2? I know first execution does, but I would like to see it there before running my actual other tasks #2018-11-01 19:45:32 richiardiandrea can I just do clojure -Sdescribe? #2018-11-01 19:45:41 richiardiandrea actually let me try locally #2018-11-01 19:46:18 richiardiandrea yep that works #2018-11-01 19:46:56 richiardiandrea $ mv .m2 .m2-backup
$ clojure -Sdescribe
Downloading: org/clojure/clojure/1.10.0-alpha6/clojure-1.10.0-alpha6.pom from
Downloading: org/clojure/spec.alpha/0.2.168/spec.alpha-0.2.168.pom from
Downloading: org/clojure/pom.contrib/0.2.2/pom.contrib-0.2.2.pom from
Downloading: org/clojure/core.specs.alpha/0.2.36/core.specs.alpha-0.2.36.pom from
Downloading: org/clojure/clojure/1.10.0-alpha6/clojure-1.10.0-alpha6.jar from
Downloading: org/clojure/spec.alpha/0.2.168/spec.alpha-0.2.168.jar from
Downloading: org/clojure/core.specs.alpha/0.2.36/core.specs.alpha-0.2.36.jar from
{:version "1.9.0.397"
:config-files ["/home/arichiardi/.local/lib/clojure/deps.edn" "/home/arichiardi/.clojure/deps.edn" ]
:install-dir "/home/arichiardi/.local/lib/clojure"
:config-dir "/home/arichiardi/.clojure"
:cache-dir "/home/arichiardi/.clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
$ ll ~/.m2
total 24
drwxrwxr-x 3 arichiardi arichiardi 4096 Nov 1 12:45 ./
drwxr-xr-x 120 arichiardi arichiardi 16384 Nov 1 12:45 ../
drwxrwxr-x 3 arichiardi arichiardi 4096 Nov 1 12:45 repository/
#2018-11-01 19:49:29 richiardiandrea it is downloading alpha6 by default #2018-11-01 20:24:14 alexmiller No, it’s not - that’s an artifact of your deps.edn files #2018-11-01 20:27:47 alexmiller By default, clj will use 1.9.0 based on the ~/.clojure/deps.edn #2018-11-01 20:46:55 richiardiandrea right #2018-11-01 20:47:10 richiardiandrea I should have use -Srepo maybe? #2018-11-01 20:47:53 richiardiandrea another question I have is if I can have aliases in parent dir and consume in children dir, in a monorepo setup for instance #2018-11-01 21:05:26 richiardiandrea I tried with CLJ_CONFIG to the root but I don't see it working... #2018-11-01 21:06:21 richiardiandrea oh now I see it working cool #2018-11-02 01:03:08 richiardiandrea How long does builds take for you folks in average? I am being told node builds take 5-10 minutes and I want to make sure I am not doing anything wrong before answering. At the moment a monorepo build takes 22 minutes and it downloads ~50 deps #2018-11-02 04:06:18 richiardiandrea This is more of a #clojurescript question actually #2018-11-02 07:42:00 slipset First of all, I know tools.deps/clj is not meant as a build tool. But still:
We now have a bunch of utils like pack, Eastwood, depot, testrunner, etc which are useful tools to use in a build pipeline.
I know I can add aliases to all of these in my home-directory’s deps.edn, but that doesn’t really work across multiple machines.
Having all the aliases duplicated across multiple projects doesn’t seem like a great solution either.
One solution I could think of is that deps.edn could refer to other deps.edns and use their aliases. Would that be possible? #2018-11-02 09:19:41 Christian Johansen what exactly is the problem in duplicating between projects? lack of standardization? #2018-11-02 09:20:43 Christian Johansen I have stuff like this: #2018-11-02 09:20:46 Christian Johansen :ancient {:extra-deps {olical/depot {:mvn/version "1.2.0"}}
:main-opts ["-m" "depot.outdated.main"]}
#2018-11-02 09:21:05 Christian Johansen you want to make that implicit in your deps.edn? #2018-11-02 09:39:13 slipset I was imagining that when I wanted to bump the version on olical/depot, I’d have to check out all my projects and bumpt it. #2018-11-02 09:39:34 slipset Also, some of the aliases are somewhat more involved than this. #2018-11-02 09:41:58 Christian Johansen yup. it’s a difficult trade-off. it’s tempting to standardize or “dedupe” some of this boilerplate, but that creates problems when things aren’t as equal anymore #2018-11-02 09:42:03 Christian Johansen e.g. you loose the flexibility #2018-11-02 09:43:26 slipset I’ve come to realise this. But that’s another discussion we could have over a cup of coffee IRL 🙂 #2018-11-02 09:43:53 Christian Johansen if you have a lot of very similar projects I guess you could make your own dev package, and encode the aliases in it, and then use it like so: #2018-11-02 09:43:57 Christian Johansen {:aliases
:dev {:extra-deps {my.company/my-bundle {:mvn/version "1234"}}}
:test {:main-opts ["my.bundle.test"]}
:ancient {:main-opts ["my.bundle.ancient"]}
:build {:main-opts ["my.bundle.build"]}}
#2018-11-02 09:44:03 Christian Johansen yes, for sure 🙂 #2018-11-02 09:45:21 Christian Johansen but that approach does quickly leads to entangled framework world, use with caution #2018-11-02 09:48:06 slipset I guess so. #2018-11-02 09:50:09 Christian Johansen I don’t have enough similar projects to warrant the effort, but this is an approach I would consider to contain the duplication and lightly enforce some standardization. you’d still have to bump the shared dependency in all projects, but at least you reduced n deps/aliases to 1 #2018-11-02 09:52:29 Christian Johansen I’m sure there are other ways to approach this as well #2018-11-02 10:02:04 orestis One thing I noticed really is that when most of those things are just standalone utilities, they shouldn’t even be in my main deps.edn file, because they don’t need all my “production” stuff to run. #2018-11-02 12:34:08 alexmiller Exactly - put them in your ~/.clojure/deps.edn #2018-11-02 13:29:14 orestis Is there then a way for the clj to ignore any local deps.edn? #2018-11-02 13:30:16 orestis Meaning, when the Classpath is created, it only has the stuff from ~/.clojure/deps.edn and nothing from a local deps.edn file. #2018-11-02 13:48:03 alexmiller no, why would you want that? #2018-11-02 14:13:06 orestis In the context of a standalone library, e.g. clj-new, cljfmt etc etc that does “build tool” stuff — it has no need to bring in the application-level deps. I’m not sure what the ramifications of having more things in the classpath are though — apart from the potential version conflicts which I think are addressable right now. #2018-11-02 10:03:47 orestis I’m always thinking that perhaps it’s time for a new “dev-time” tool that provides a wrapper and some porcelain around those single-purpose libraries. Sure you lose the flexibility, but you also don’t have to make all those decisions when you are a newcomer to the ecosystem, plus the single-purpose libraries can still be used standalone and can evolve at their own pace. #2018-11-02 10:09:51 Christian Johansen wherever it is, all the deps and scripts required to build and deploy the app should reside in the repo at least #2018-11-02 10:09:51 dominicm @orestis would such a thing be put into each project? Or be part of the setup guide for ~/.clojure/deps.edn when you're new to a project? #2018-11-02 10:11:07 Christian Johansen I strongly believe that developing any app should require as few pre-installed tools as possible, and that the code/repo should be as self-contained as possible #2018-11-02 12:27:46 rickmoynihan I agree with this sentiment too. tools.deps is quite good here, as you can effectively ship tooling with a repo; and version lock the tools - without having to include binaries etc. #2018-11-02 10:11:26 orestis I think it could be its own standalone CLI, like lein or boot. #2018-11-02 10:12:05 orestis But in the end all it would do is generate the correct clj/clojure invocation for you. #2018-11-02 10:13:32 Christian Johansen I’m currently using deps.edn + Makefile to get the job done. I see some things that will probably seem like tedious repetition at some point, but so far I’m loving the simplicity of it. no magic, you can see everything #2018-11-02 10:13:32 orestis I see your point though @christian767 — though with tools.deps you need to have clj installed — otherwise, well, you have to fallback to java -cp ..., no? #2018-11-02 10:14:00 Christian Johansen yes, I think clj and java is a good baseline requirement for clojure projects at this point #2018-11-02 10:14:09 orestis Right, a more opinionated Makefile is what I’m trying to describe. #2018-11-02 10:14:18 Christian Johansen sure #2018-11-02 10:14:58 orestis My experience in trying to setup a new deps.edn based project was a few days trying out the various uberjar libraries, then a linter, then a formatter, then an outdated-deps-finder etc etc etc. #2018-11-02 10:15:29 orestis I think the community would benefit if this is done for you, while still maintaining the much-needed transparency and zero magic. #2018-11-02 10:15:46 Christian Johansen I agree #2018-11-02 10:16:58 orestis I wonder how could this move forward? Ask people to upload their deps.edn and makefiles and shell scripts somewhere and compare approaches? I’m too much of a newb to do this based only on my assumptions, and different projects will have wildly different needs. #2018-11-02 10:19:08 dominicm You start a project by cloning edge, and then modifying it. Everything is set up for you. clj is all you need. #2018-11-02 10:19:29 dominicm Templates also solve this problem. #2018-11-02 10:20:26 Christian Johansen basically, scaffolding #2018-11-02 10:21:01 dominicm I wouldn't say scaffolding is a good comparison. It's more of a cookie cutter or starting point than anything. #2018-11-02 10:21:02 orestis The issue with templates/scaffolding is that after the initial project creation, you are on your own. #2018-11-02 10:21:16 Christian Johansen my issue with this approach is that it sets you off for an overwhelming start, and it is too easy to end up with a bunch of stuff laying around that you don’t really need #2018-11-02 10:21:34 dominicm @orestis that's why we suggest cloning. You can upgrade by doing git merge edge/master #2018-11-02 10:22:03 dominicm I've been making some adjustments to reduce the chances of conflicts to very low. It should only happen if you modify things (in which case, what do you expect?) #2018-11-02 10:22:32 dominicm The documentation around this isn't great, but it's how we expect edge to be used, and how we use it internally. #2018-11-02 10:24:29 dominicm blugh, s/modify things/modify shared areas in a separate folder/ #2018-11-02 10:26:13 orestis I see your point — and it might be hugely valuable for a consulting company, since you tend to start projects very often, right? I’m more after something that decouples some very common concerns for most people, while imposing no opinions on how your actual app is structured. Meaning, I just want to run a command to give me outdated dependencies, a command to run all my tests, a command to build my uberjar. I’d like newcomers to the community to have a single entry point, and if/when they want more power, be able to drop down to the layer below. #2018-11-02 10:27:04 orestis Crucially, they shouldn’t have to know the underlying libraries if they don’t care. Similarly to how you don’t really know how lein builds uberjars. #2018-11-02 10:27:19 dominicm I know 🙈 You don't want to #2018-11-02 10:27:47 orestis And also crucially though, the uberjar building library is standalone and people can use it as they see fit. #2018-11-02 10:27:57 Christian Johansen koacha does this for tests #2018-11-02 10:28:15 Christian Johansen you could create a binstub on your box, run koacha in your app, and it’ll “just work” #2018-11-02 10:28:31 dominicm We start, and maintain, a lot of projects, yep 🙂.
edge is slightly more opinionated than that, it assumes integrant for example. But I think the implementation would work for this? Use a git repo, and merge periodically to have the uberjar builder updated by maintainers upstream. #2018-11-02 10:29:58 orestis Exactly @christian767, kaocha is very good at that, it has its own CLI - like thing, but you can also use its API if you want to. #2018-11-02 10:30:13 Christian Johansen yup #2018-11-02 10:31:23 dominicm Binstubs are only part of the puzzle though 🙂
What would provide:
> I think the community would benefit if this is done for you, while still maintaining the much-needed transparency and zero magic.
? #2018-11-02 10:31:24 orestis So what I’m after is — a cli tool that you could install, then run XXX test and it would do the right thing, using kaocha under the hood. Kaocha then can keep evolving and people can use it standalone without the XXX tool. #2018-11-02 10:32:50 orestis You wouldn’t know that kaocha is used unless you really want to — kaocha is a very good example since it plays well with clojure.test. Uberjars can be more fiddly to configure, I guess? #2018-11-02 10:33:27 slipset This is sort of the discussion I was looking for #2018-11-02 10:33:46 slipset But in one way, I guess we’re discussing yet another lein or boot #2018-11-02 10:33:49 dominicm jars need a lot of configuration in my experience 🙂 Clients come up with all sorts of novel environments in which we have to deploy. #2018-11-02 10:33:59 dominicm @slipset I wouldn't like to see another lein or boot fwiw. #2018-11-02 10:34:46 slipset I know, but the ideas I’m sort of looking for very quickly lead in that direction, unfortunatly #2018-11-02 10:35:17 slipset Even if it’s only @christian767 Makefile or Juxt’s edge. #2018-11-02 10:35:36 slipset I could imagine ending up in: #2018-11-02 10:36:21 slipset $ make -f /usr/local/etc/default-makefile-for-app setup #2018-11-02 10:36:56 slipset which is basically lein new app or clj -A:clj-new app ... #2018-11-02 10:36:59 slipset or some sort. #2018-11-02 10:38:10 orestis I’m not 100% sure if this only an “ergonomics” thing — or if some plumbing is still missing from tools.deps. #2018-11-02 10:39:37 dominicm Given Stu's advocacy for CLJ-2373 (I've memorized it he's said it so much!), let's take a leaf out of the Clojure/core approach to solving problems.
What is the problem statement here? #2018-11-02 10:40:46 dominicm If we were only trying to solve the assembly problem, then templates are fine. So there's another element too I'm guessing? #2018-11-02 10:46:38 dominicm I kinda get the impression we're each looking for different additional properties. I've seen mention of "project-local isolation", "baseline as clj", and "not caring about underlying library". #2018-11-02 11:02:34 dominicm fwiw, I care more about "baseline as clj" because it's the minimum I can expect anyone to have done, ever. If we're doing training as a company, getting binstubs installed, covering all the distros is more hassle than it's worth for me.
I would probably find it acceptable if a project had some kind of alias like clj -A:tool-name which would then do the rest for itself. add-lib is pretty compelling here, as the tool can dynamically fetch it's dependencies as needed. #2018-11-03 06:02:14 seancorfield @alexmiller merged master back to the add-lib branch and we just added a :deps alias at work to that SHA so we can use add-lib in our REPLs. It's nice. I need update my dot-clojure file/repo with that too. #2018-11-02 12:13:34 orestis Nice breakdown. Fully agree. #2018-11-02 16:04:12 seancorfield As for a self-contained project: yes, very important -- it's why we have a shell wrapper that sets CLJ_CONFIG (to point to a repo-specific "dot Clojure" folder) and we have a deps.edn file for each subpar object in the repo. #2018-11-02 16:26:08 richiardiandrea Funny I was doing the same yesterday but it seemed like a hack ^ #2018-11-02 16:27:36 andy.fingerhut But today, it no longer seems like you are hacking. Today, you are corfielding 🙂 #2018-11-02 16:28:42 richiardiandrea It seems like everybody is going to corfield their deps.edn sooner or later 😄 #2018-11-02 16:29:19 andy.fingerhut Sean, if this goes seriously weird, I sincerely apologize #2018-11-02 16:44:02 andy.fingerhut If it only goes slightly weird, then .... you are welcome? 🙂 #2018-11-02 18:07:20 seancorfield Weird is good. I like weird. :rolling_on_the_floor_laughing: #2018-11-02 18:10:36 seancorfield Back to the CLJ_CONFIG stuff (I was on my phone earlier)... we've found the ability to specify a "home" deps.edn via CLJ_CONFIG is great! It means a project can have a "home" configuration for aliases that provide project-specific tooling as well as :override-deps (which we rely on heavily to "pin" versions across subprojects within the overall project. #2018-11-02 18:13:03 seancorfield I talked a bit our shell wrapper a week or two back I think. Mostly it takes commands of the form build alias(es) subproject and it goes into that subproject and runs CLJ_CONFIG=../versions clojure -A:defaults:aliases (edited to add our #2018-11-02 18:19:51 alexmiller Is that only for aliases? #2018-11-02 18:19:51 seancorfield It does a few mini-DSL-y sort of things around plural aliases or aliases with certain prefix/suffix strings as well as expanding some internal "aliases" but it's only a few tens of lines of shell script. #2018-11-02 18:20:53 seancorfield @alexmiller We have the "pinned" version of Clojure specified as a regular :deps in that versions/deps.edn file but everything else is just aliases I think. #2018-11-02 18:23:31 seancorfield Oh, and it adds our Archiva Maven repo as well. #2018-11-02 18:27:23 seancorfield It's definitely shown the need for all deps-based tooling to support correct deps.edn merging with support for :override-deps and aliases, if it doesn't just execute in the context of your project's classpath (Depot, for example, already supports aliases but needed a PR to support :override-deps merging). #2018-11-02 18:27:42 seancorfield It would definitely simplify usage if :override-deps could be top-level and not need an alias. #2018-11-02 19:06:13 dominicm @seancorfield fwiw, I don't like the CLJ_CONFIG solution because it takes away the ability to have aliases (e.g. A custom repl) from a user.
I understand your choice and think it is pragmatic though. #2018-11-02 19:40:36 seancorfield Yeah, it's a compromise. We have REPL-related aliases in that deps file, as well as other tooling. #2018-11-02 19:41:13 seancorfield What I'd really like would be the ability to specify an extra deps.edn file easily between user-deps and project-deps. #2018-11-02 23:36:39 seancorfield @alexmiller Is this something you'd entertain as an enhancement? Essentially to support something like <system>/deps.edn, <user>/deps.edn, <tooling>/deps.edn, (project) deps.edn. #2018-11-02 23:37:13 seancorfield I wasn't sure if you just missed this in the flow of questions in the channel since you didn't comment -- and I sort of expected you to 🙂 #2018-11-03 00:04:38 alexmiller Didn’t see it #2018-11-03 00:05:41 alexmiller Don’t know - have been hoping to move towards fewer deps.edn files, not more :) #2018-11-03 00:08:20 seancorfield Fair enough. The underlying t.d.a. lib supports an arbitrary number of deps.edn files tho'... and it would be nice and generic to surface that in clj/`clojure` 🙂 #2018-11-03 00:09:56 seancorfield You can easily merge extra stuff at the end with -Sdeps but you can't insert anything before the project deps.edn which is where I find the need -- and I think I'm not alone. #2018-11-03 00:10:38 seancorfield We can get away with "replacing" the user-deps via CLJ_CONFIG for our needs but it does mean some duplication and inconvenience. #2018-11-03 00:28:12 alexmiller Other things are pointing away from having an arbitrary number of deps so that’s much more likely to die #2018-11-03 00:28:52 alexmiller But I asked because I’m interested in the use cases #2018-11-03 00:28:53 seancorfield Interesting. I wonder how we'll make things work if that goes away. It's kind of fundamental to our workflow right now. #2018-11-03 00:32:24 seancorfield I guess we'd need to maintain a variant of the clojure script that allowed us to specify an extra deps file around these lines # Chain deps.edn in config paths. repro=skip config dir
if "$repro"; then
config_paths=("$install_dir/deps.edn" "deps.edn")
else
config_paths=("$install_dir/deps.edn" "$config_dir/deps.edn" "deps.edn")
fi
config_str=$(printf ",%s" "${config_paths[@]}")
config_str=${config_str:1}
#2018-11-03 00:34:44 seancorfield (which part are you saying might "die"? support for CLJ_CONFIG or support for the --config-files command line argument in t.d.a.?) #2018-11-02 19:41:52 seancorfield So we'd have <system>/deps.edn, <user>/deps.edn, <tooling>/deps.edn, (project) deps.edn. #2018-11-02 19:42:27 timgilbert Say, I'm getting this error trying to mvn install a tools.deps.alpha repo I've got checked out locally, anyone know if there is some special stuff I need to do in order to build it?
[INFO] --- clojure-maven-plugin:1.7.1:test (clojure-test) @ tools.deps.alpha ---
Error: Could not find or load main class clojure.main
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
#2018-11-02 19:46:13 alexmiller Can you provide more info? Is there a pom? Did you gen it or make it? What do you mean by a “tools.deps.alpha repo”? #2018-11-02 19:47:48 timgilbert Sure. I just cloned https://github.com/clojure/tools.deps.alpha and ran mvn install. There is a pom, which I've left unchanged from the repo. I couldn't find any docs about a build process, but I'm sure I could have missed something. #2018-11-02 19:49:19 timgilbert (I get the same error running mvn test, which makes sense) #2018-11-02 19:50:30 timgilbert Here is the full output:
% mvn install
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 1 modules...
[INFO] Installing Nexus Staging features:
[INFO] ... total of 1 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building tools.deps.alpha 0.5.461-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-source (add-clojure-source-dirs) @ tools.deps.alpha ---
[INFO] Source directory: /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/main/clojure added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-resource (add-clojure-source-dirs) @ tools.deps.alpha ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-test-source (add-clojure-test-source-dirs) @ tools.deps.alpha ---
[INFO] Test Source directory: /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/test/clojure added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.12:add-test-resource (add-clojure-test-source-dirs) @ tools.deps.alpha ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ tools.deps.alpha ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/main/resources
[INFO] Copying 18 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.0:compile (default-compile) @ tools.deps.alpha ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ tools.deps.alpha ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /mnt/c/Users/tim/Dropbox/src/cljs/tools.deps.alpha/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.0:testCompile (default-testCompile) @ tools.deps.alpha ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ tools.deps.alpha ---
[INFO]
[INFO] --- clojure-maven-plugin:1.7.1:test (clojure-test) @ tools.deps.alpha ---
Error: Could not find or load main class clojure.main
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.073 s
[INFO] Finished at: 2018-11-02T19:48:28+00:00
[INFO] Final Memory: 19M/295M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.theoryinpractise:clojure-maven-plugin:1.7.1:test (clojure-test) on project tools.deps.alpha: Clojure failed. -> [Help 1]
#2018-11-02 19:51:29 timgilbert Sorry, I could put this all in a ticket too. Just thought I might be missing something obvious. #2018-11-02 19:52:19 alexmiller Ok, so you actually mean tda #2018-11-02 19:52:49 alexmiller No magic here, it is just a maven build and I run mvn test on it all the time #2018-11-02 19:52:56 alexmiller What java version? #2018-11-02 19:53:35 alexmiller Prob needs 8+ #2018-11-02 19:57:01 timgilbert % java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-1ubuntu0.16.04.1-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
% mvn --version
Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.8.0_181, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-17134-microsoft", arch: "amd64", family: "unix"
#2018-11-02 19:57:55 timgilbert This is running under ubuntu in the "windows subsystem for linux" which is a bit of a weird environment but all the usual clojure stuff works fine #2018-11-02 19:57:58 alexmiller I’m using Java 11 and Maven 3.5.4 at the moment, but I don’t have any reason to believe your versions are not sufficient #2018-11-02 19:58:14 alexmiller the build box is certainly using much older stuff to build and deploy that lib #2018-11-02 19:58:27 alexmiller works on my machine :) #2018-11-02 19:58:30 alexmiller and the build box :) #2018-11-02 19:58:47 alexmiller try throwing -X -e on your mvn command #2018-11-02 20:00:29 timgilbert I was wondering if I might need to build clojure locally first or something #2018-11-02 20:00:50 alexmiller nope #2018-11-02 20:01:01 alexmiller it’s just using the released clojure 1.9.0 #2018-11-02 20:01:43 timgilbert Here's the exception with -X -e:
[ERROR] Failed to execute goal com.theoryinpractise:clojure-maven-plugin:1.7.1:test (clojure-test) on project tools.deps.alpha: Clojure failed. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.theoryinpractise:clojure-maven-plugin:1.7.1:test (clojure-test) on project tools.deps.alpha: Clojure failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Clojure failed.
at com.theoryinpractise.clojure.AbstractClojureCompilerMojo.callClojureWith(AbstractClojureCompilerMojo.java:464)
at com.theoryinpractise.clojure.AbstractClojureCompilerMojo.callClojureWith(AbstractClojureCompilerMojo.java:366)
at com.theoryinpractise.clojure.ClojureRunTestWithJUnitMojo.execute(ClojureRunTestWithJUnitMojo.java:138)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
#2018-11-02 20:02:52 alexmiller It’s failing in the test step, which is kinda weird. You could use -Dmaven.test.skip=true if you wanted to just move past that #2018-11-02 20:03:48 timgilbert Aha! That did it. #2018-11-02 20:03:54 alexmiller it’s failing to run Clojure to run the tests. not sure why that would be #2018-11-02 20:04:11 alexmiller I’ve seen a lot of maven build failures, but I haven’t seen this one :) #2018-11-02 20:04:13 timgilbert My next question was going to be how to run the tests 😉 #2018-11-02 20:04:47 timgilbert It's possible that some local modifications I made messed something up, let me try on a clean master #2018-11-02 20:09:27 timgilbert Hmm, still getting the same thing on a clean master. I'll see if I can repo at home on my mac and file a ticket if so. Thanks for looking. #2018-11-02 20:17:34 seancorfield It works on my Mac. I haven't tried to build/install any Clojure Contrib projects on WSL on my Windows 10 laptop. I'll try to remember to test that later today. #2018-11-02 20:55:33 timgilbert The root of the problem seems to be that I'm getting the clojure dependency with scope provided.
% mvn dependency:tree | grep org.clojure
[INFO] org.clojure:tools.deps.alpha:jar:0.5.461-SNAPSHOT
[INFO] +- org.clojure:clojure:jar:1.9.0:provided
[INFO] | +- org.clojure:spec.alpha:jar:0.1.143:provided
[INFO] | \- org.clojure:core.specs.alpha:jar:0.1.24:provided
[INFO] +- org.clojure:data.xml:jar:0.2.0-alpha5:compile
[INFO] | \- org.clojure:data.codec:jar:0.1.0:compile
[INFO] +- org.clojure:tools.gitlibs:jar:0.2.64:compile
[INFO] \- org.clojure:tools.cli:jar:0.3.5:compile
#2018-11-02 20:56:25 timgilbert Anyways, I'll mess around a bit more and put a ticket up. I'm guessing this is something local to my machine. #2018-11-02 20:56:34 alexmiller interesting. I wonder why that’s different here vs there #2018-11-02 21:12:38 seancorfield @U08QZ7Y5S FYI, if you're relying on WSL, avoid Insider build 18272 -- they broke WSL in that build (they know about it, not sure when a fix will flight tho'). #2018-11-02 21:13:27 timgilbert After the October fiasco, I'm avoiding Windows updates altogether for some time #2018-11-02 21:13:44 alexmiller October Fiasco is my new hardcore band btw #2018-11-02 21:20:02 seancorfield I'm on Fast Ring Insider builds so my WSL is bricked on both my Windows machines. Fortunately, I still have my Mac for dev work! 🙂 #2018-11-07 17:20:39 alexmiller @U08QZ7Y5S I believe you’re experiencing this, and I feel bad about that, but priority-wise has to fall into my “won’t fix” bucket, so I’m going to decline it. If you manage to find a fix, happy to see it re-opened and consider. #2018-11-07 17:25:44 timgilbert That's totally fine with me, I mostly just wanted to get it documented in case somebody else was seeing it. I have to confess that I have no idea how mvn test ever works without a concrete clojure.jar in the dependency tree, though. #2018-11-07 17:33:39 alexmiller prob via the clojure maven plugin (which is in the classpath) #2018-11-02 19:44:27 john What are people using as an environ replacement? Does it only work with a project.clj? #2018-11-02 19:47:55 seancorfield We have our own, internal, configuration library that can use env vars but mostly relies on EDN files. #2018-11-02 23:05:57 dominicm I'd enjoy evangelizing aero to you 🙂 Any big differences between aero and your system? #2018-11-02 23:34:02 seancorfield I'd have to go look at Aero again. We did a review of pretty much everything available out there when we decided to "roll our own" a while back... #2018-11-03 11:09:54 yogidevbear (whispers "aero" into Sean's ear while he's sleeping) #2018-11-02 19:48:33 seancorfield We've considered open sourcing it, but it does contain some proprietary stuff that we haven't had time (or inclination) to refactor out yet. #2018-11-02 20:04:52 kenny If I have this in my :deps:
compute/integrations {:git/url "
And I run clj -Stree -Sforce, I get this:
compute/integrations
Why is commons-logging/commons-logging included? #2018-11-02 20:07:27 alexmiller dunno, that’s weird #2018-11-02 20:07:46 alexmiller feel free to file a ticket with a repro if you can slim it down #2018-11-02 20:08:11 alexmiller it being under a git dep might be part of the answer, not sure #2018-11-02 20:08:58 kenny Will see if I can get a repro. #2018-11-02 20:51:36 kenny First cut at the obvious repro case and I cannot reproduce. I made a git repo with a deps.edn that has clj-http as the dep. I made directory with a deps.edn on my computer that added that lib as a dep with the commons-logging exclusion and it behaved correctly. Is there something else I should be looking for in my current problem case that may shed some light here? #2018-11-02 20:53:46 kenny If I add that exact dependency to my new project:
{:deps {compute/integrations {:git/url "
commons-logging is not in the tree, as expected. #2018-11-02 20:55:13 kenny #2018-11-02 21:00:30 kenny This dep excludes httpclient, as expected.
compute/integrations {:git/url " #2018-11-02 21:01:28 kenny Is there a way to see all the deps that have commons-logging as a dep? #2018-11-02 21:03:50 kenny Removing all other deps in my project besides Clojure and compute/integrations causes the exclusion to work correctly. #2018-11-02 21:39:52 kenny Got a repro. There is something seriously funky going on. Repro is here: https://github.com/kennyjwilli/clj-exclusions. cd into the app directory and run clj -Stree -Sforce. I get this:
org.clojure/clojure 1.9.0
org.clojure/core.specs.alpha 0.1.24
org.clojure/spec.alpha 0.1.143
integrations/integrations ..
clj-http/clj-http 3.9.1
commons-codec/commons-codec 1.11
org.apache.httpcomponents/httpasyncclient 4.1.3
org.apache.httpcomponents/httpcore-nio 4.4.6
slingshot/slingshot 0.12.2
commons-io/commons-io 2.6
org.apache.httpcomponents/httpcore 4.4.9
org.apache.httpcomponents/httpclient-cache 4.5.5
org.apache.httpcomponents/httpclient 4.5.5
commons-logging/commons-logging 1.2
potemkin/potemkin 0.4.5
clj-tuple/clj-tuple 0.2.2
riddley/riddley 0.1.12
org.apache.httpcomponents/httpmime 4.5.5
com.amazonaws/aws-java-sdk-cloudwatch 1.11.399
com.amazonaws/jmespath-java 1.11.399
com.fasterxml.jackson.core/jackson-databind 2.6.7.1
com.fasterxml.jackson.core/jackson-core 2.6.7
com.fasterxml.jackson.core/jackson-annotations 2.6.0
com.amazonaws/aws-java-sdk-core 1.11.399
joda-time/joda-time 2.8.1
com.fasterxml.jackson.dataformat/jackson-dataformat-cbor 2.6.7
software.amazon.ion/ion-java 1.0.2
Now if I change the name of the dependency from integrations to lib, I get this output from clj -Stree -Sforce:
org.clojure/clojure 1.9.0
org.clojure/core.specs.alpha 0.1.24
org.clojure/spec.alpha 0.1.143
com.amazonaws/aws-java-sdk-cloudwatch 1.11.399
com.amazonaws/jmespath-java 1.11.399
com.fasterxml.jackson.core/jackson-databind 2.6.7.1
com.fasterxml.jackson.core/jackson-core 2.6.7
com.fasterxml.jackson.core/jackson-annotations 2.6.0
com.amazonaws/aws-java-sdk-core 1.11.399
joda-time/joda-time 2.8.1
com.fasterxml.jackson.dataformat/jackson-dataformat-cbor 2.6.7
software.amazon.ion/ion-java 1.0.2
org.apache.httpcomponents/httpclient 4.5.5
commons-logging/commons-logging 1.2
lib/lib ..
clj-http/clj-http 3.9.1
commons-codec/commons-codec 1.11
org.apache.httpcomponents/httpasyncclient 4.1.3
org.apache.httpcomponents/httpcore-nio 4.4.6
slingshot/slingshot 0.12.2
commons-io/commons-io 2.6
org.apache.httpcomponents/httpcore 4.4.9
org.apache.httpcomponents/httpclient-cache 4.5.5
potemkin/potemkin 0.4.5
clj-tuple/clj-tuple 0.2.2
riddley/riddley 0.1.12
org.apache.httpcomponents/httpmime 4.5.5
It is different and all I did was change the name. What is going on here? #2018-11-02 21:52:59 alexmiller the ordering is somewhat dependent on traversal order as to which one is encountered first, but you shouldn’t be getting a dep that is excluded #2018-11-02 21:53:32 alexmiller there are a couple tickets out there that I’ve looked at and there are a couple issues to resolve - it’s likely this is the same as one of them #2018-11-02 21:54:51 kenny Ok. Which tickets? #2018-11-02 21:55:17 seancorfield They're only different because org.apache.httpcomponents/httpclient is coming in via two dependencies, not just one. #2018-11-02 21:56:28 seancorfield In the integrations case, httpclient and logging come in via clj-http. In the lib case, httpclient and logging are coming in via aws cloudwatch (via aws core) #2018-11-02 21:57:29 kenny The inconsistency is quite misleading though. #2018-11-02 21:58:22 alexmiller TDEPS-58 (and TDEPS-79) #2018-11-02 21:59:11 kenny Yep that sounds like it. #2018-11-02 21:58:23 seancorfield If you have the exclusion on clj-http, it won't stop it being pulled in if the cloudwatch dep is walked first -- since it won't have seen the exclusion at that point. #2018-11-02 21:59:15 seancorfield We've had to be very aggressive with exclusions to stop up all the "holes" where certain things leak in... #2018-11-02 21:59:37 alexmiller there have been some requests for “global” exclusions #2018-11-02 21:59:51 kenny Aggressive meaning exclusions on a lot of your deps? #2018-11-02 22:00:01 alexmiller I’m not necessarily opposed to that, but haven’t thought through all the implications #2018-11-02 22:00:08 kenny I think Boot has that. #2018-11-02 22:00:46 seancorfield Yes. We've had to add the same/similar exclusions on several deps across our subprojects in order to completely "hide" them. #2018-11-02 22:01:33 kenny Would global exclusions solve your problem? #2018-11-02 22:01:52 seancorfield We've used global exclusions with Boot. It's a bit of a sledgehammer. We used it to exclude Clojure and ClojureScript across everything. #2018-11-02 22:03:04 seancorfield Yes, global exclusions would "solve" the problem but it's really a huge hammer to crack a little nut. And the fall out is that you then need to ensure you bring in all those things explicitly in the right subprojects -- which is a pain. #2018-11-02 22:03:07 alexmiller seems like that would make it hard to use Clojure or ClojureScript :) #2018-11-02 22:03:39 seancorfield You can bring them in at the top-level. It just ensures nothing else brings them in. #2018-11-02 22:05:48 alexmiller so I’d say that’s global-except-at-top-level :) #2018-11-02 22:06:58 seancorfield Yeah, well, Boot is nothing if not "pragmatic" 🙂 #2018-11-02 22:07:20 alexmiller I’m just saying, that’s a subtlety #2018-11-02 22:03:58 kenny Dependency resolution wouldn't solve that for you? #2018-11-02 22:06:03 alexmiller with Maven dependency, no #2018-11-02 22:06:32 alexmiller Maven takes the first version it encounters #2018-11-02 22:06:35 seancorfield Two things: 1. we wanted ClojureScript excluded completely for all our backend projects, 2. we run "pedantic" style checks (or, at least, we did in lein and boot) so if potentially conflicting versions came in, we would get an exception -- and we didn't want to add those exclusions to nearly every Clojure dependency, since everything depends on different versions of Clojure out there 🙂 #2018-11-02 22:11:06 kenny Anyway, thanks for the help. Excluding commons-logging from my other top level AWS deps fixed the issue. I voted and followed TDEPS-58. #2018-11-03 00:24:05 lwhorton i want to report a big-ol THANK YOU to the hard work behind tools-deps. i just started a greenfield clojure project and was able to create an nrepl “reloaded” workflow using only 1. deps.edn file 2. makefile for convenience 3. cat. no more boot, no more lein (for now, still have to look at generating jars). #2018-11-03 00:24:23 lwhorton is this what cleanliness feels like? have i found it?! troll #2018-11-03 00:25:31 alexmiller Living the dream #2018-11-03 00:28:10 seancorfield I'll second that THANK YOU since we have essentially eliminated Boot from our primary Clojure workflow across all tiers (dev through QA through production) and we like the simplicity of it all. #2018-11-03 10:28:05 borkdude @lwhorton mind sharing your deps.edn? I’m interested in enabling nREPL for a project so I don’t have to use lein #2018-11-03 16:52:34 lwhorton sure:
:aliases {:dev {:extra-paths ["dev"]
:extra-deps {org.clojure/tools.namespace {:mvn/version "0.2.11"}
org.clojure/tools.deps.alpha {:mvn/version "0.5.460"}
nrepl/nrepl {:mvn/version "0.4.5"}
reply {:mvn/version "0.4.2"}}
:main-opts ["-m" "nrepl.cmdline"]}}}
#2018-11-03 16:52:39 lwhorton makefile:
NREPL_PORT=$(shell cat .nrepl-port)
dev:
clj -A:dev
.PHONY: dev
clj-repl:
clj -R:dev -m reply.main --attach $(NREPL_PORT)
#2018-11-03 16:55:17 lwhorton and my user.clj file (pulled in from the extra-paths in deps.edn):
(ns user
(:require
[ :as io]
[clojure.tools.namespace.repl :refer [refresh refresh-all]]
[clojure.pprint :refer [pprint]]
[clojure.inspector :as i]
[gerbil.core :as c]
)
)
(def ^:dynamic system nil)
(defn init []
(alter-var-root #'system (constantly (c/system))))
(defn start []
(alter-var-root #'system c/start))
(defn stop []
(alter-var-root #'system
(fn [s] (when s (c/stop s)))))
(defn go []
(init)
(start))
(defn reset []
(stop)
(refresh :after 'user/go))
#2018-11-03 16:56:05 lwhorton you can drop the reply stuff if you want and just use nrepl’s “basic” client #2018-11-03 11:26:29 Christian Johansen I'll third that THANK YOU! :+1: #2018-11-03 12:10:51 slipset @lwhorton juxt pack generates jars for you #2018-11-03 15:56:24 orestis @borkdude :cider-nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.4.5"}
cider/cider-nrepl {:mvn/version "0.18.0"}}
:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]}
:nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.4.5"}
org.clojure/tools.nrepl {:mvn/version "0.2.13"}}
:main-opts ["-m" "nrepl.cmdline"]} #2018-11-03 15:56:51 borkdude thanks #2018-11-03 15:57:11 orestis Here’s what I use to launch nREPL — the first variant is what I actually use to connect CIDER. #2018-11-03 16:11:14 cap10morgan Asking this here too since it’s not really Datomic-specific: I have a datomic-pro download in a dir (so it has the datomic-pro-version.jar at the root, a lib subdir with deps jars, etc. I am trying to setup a tools-deps project to run some clojure code using datomic and all of its deps w/o triggering a download of them. Is there a more efficient way to do it than having to specify each individual jar in the :paths vector in deps.edn? #2018-11-03 16:29:05 cap10morgan Answering my own question:
:paths ["src" "/path/to/datomic/datomic-pro-version.jar" "/path/to/datomic/lib/*"] was the answer. (Not "/path/to/datomic/lib/" nor "/path/to/datomic/lib/*.jar") #2018-11-05 06:57:56 slipset Reason I’m asking is that I’m pondering creating a deps-shipit which would generate/refresh the pom, update the versions in it, create your choice of (skinny/uber) jar and then upload it to Clojars. #2018-11-05 07:58:34 dominicm Uberjars aren't really suitable for uploading to clojars #2018-11-05 08:07:20 slipset true enough #2018-11-05 08:07:59 dominicm @slipset fwiw, I'd be interesting in a tool which only concerns itself with uploading jars to maven repos. Maybe with support for s3 wagons too. #2018-11-05 08:09:20 dominicm Having a tool which starts a JVM to run this shell script isn't immediately on my list of problems I need solving:
#!/bin/sh
clojure -A:pack -m …
clojure -Spom
clojure -A:shipit output.jar pom.xml
#2018-11-05 08:10:41 dominicm This is fantastic ❤️ 2 comments:
1. Can we look at getting gpg support for credentials, I think this is really important. Environment variables are far from secure.
2. I'd love to have this replace my current documentation in pack for using mvn:deploy-file 🙂 #2018-11-05 08:11:24 slipset 1) Yes. If I could ask you to file an issue/submit PR, that’d be fantastic #2018-11-05 08:11:28 dominicm I asked for this to exist about a week ago, so I'm glad it now does. I'm going to have to try imagining more things. Pile of Gold. Pile of Gold. #2018-11-05 08:12:04 slipset 2) I’m more than happy to improve it in any way 🙂 #2018-11-05 08:12:40 slipset In fact, I was about to submit a PR against pack suggesting the use of deps-deploy this morning 🙂 #2018-11-05 08:16:43 dominicm @slipset I don't think so. I am not aware of that feature at all though 🙂 #2018-11-05 08:17:33 slipset Ok, then if you could outline a solution in a ticket, I’d be very happy. I haven’t worked much with gpg. #2018-11-05 08:18:54 dominicm I've put quickly jotted down some thoughts, I've tried to capture my overarching goal of "moar secure" and provide some options. #2018-11-05 08:19:21 slipset Thank you! I’ll have a closer look this evening. #2018-11-05 08:20:48 slipset Btw, concerning a potential deps-shipit, you’d actually end up just firing up one jvm, as the plan for deps-shipit would be to compose the three steps you are outlining into one. #2018-11-05 08:23:56 dominicm I know 🙂 But it's similar to this also:
#!/usr/bin/env clojure
(require '[pack.onejar])
(require '[deploy])
(require '[t.d.a])
(pack.onejar/jar)
(t.d.a/generate-pom)
(deploy/deploy jar pom)
Having said that, I'm usually happier to automate myself than others are 🙂 #2018-11-05 08:24:07 slipset 🙂 #2018-11-05 08:28:23 dominicm Supporting jar signing seems important too, trying to find some good references on it #2018-11-05 08:29:24 slipset Shouldn’t be too hard, lein already provides this. Just need to RTFS a bit. #2018-11-05 08:36:23 dominicm yeah. Reading the source looks like the best option, given that maven + gpg signing isn't giving me anything useful. #2018-11-05 12:21:35 rickmoynihan In my experience gpg and lein has been a bit awkward to setup… and I’ve seen lots of people struggle to do it. Is there an option to use something like bouncy castle here ( https://www.bouncycastle.org/java.html )? Or perhaps something that ships with the JVM?
I’m assuming the main thing gpg brings to the table (aside from the crypto) is being able to interact with the OS keychain. #2018-11-05 12:21:56 rickmoynihan Not sure if bouncy castle has any support for that #2018-11-05 13:04:14 dominicm @rickmoynihan You've highlighted my concern with BouncyCastle immediately. It would be good to enumerate the exact issues people have. I would expect it's around gpg vs gpg2. But I'm potentially projecting the issue I personally have. #2018-11-05 13:04:41 dominicm I would expect using Bouncy Castle with PGP brings in a new class of issues, like support smart cards. #2018-11-05 13:05:19 dominicm One option is to search & present options. e.g. if only gpg/gpg2 are found, use those. If both, then ask the user. #2018-11-05 13:35:39 rickmoynihan Yeah; I suspect gpg/gpg2 are the best options… as lein and others have been through this before and everyone always seem to settle on gpg. It’s just frustrating that that choice often leaves to brittleness and installation/configuration issues… but I guess it’s always going to be a bit of a cross platform nightmare 😞 #2018-11-05 13:46:27 dominicm It doesn't have to be I think 🙂 #2018-11-05 13:46:41 dominicm Lein also doesn't prompt the user ever. There are other mistakes there. #2018-11-05 14:15:49 rickmoynihan I can’t really recall my issues with gpg… it could well have been gpg vs gpg2. IIRC mine might also have been due to config cruft from .bashrc which I ported from linux to mac years back. In my experience it’s just one of those areas which can be fiddly, where you really don’t want something fiddly. #2018-11-05 14:59:12 dominicm If gpg is hard to use, I'd be inclined to not tackle the problem 🙂 #2018-11-05 14:59:15 dominicm But that's just me. #2018-11-05 14:59:49 dominicm What I mean is, if gpg is easy to shoot yourself in the foot with, that's gpg's problem. I don't think it's a good use of time to handle the thousands of config combinations. #2018-11-05 15:30:06 slipset lein actually prompts me for my gpg thing (if I don’t turn off the signed artifacts thingy) #2018-11-05 15:43:36 dominicm @slipset your gpg passphrase you mean? #2018-11-05 16:00:42 slipset yes, I guess that’s the correct term. #2018-11-05 16:04:34 seancorfield After some updates to software on my Mac I could never get gpg working again so I just told leiningen not to sign things. I mostly use boot to upload to clojars to avoid gpg anyway. #2018-11-05 16:06:32 seancorfield And I have my clojars credentials in my dot file unencyrpted - again so I can avoid trying to deal with gpg. #2018-11-05 16:06:58 seancorfield (And I was never able to get anything to work on windows in this area) #2018-11-05 16:08:16 seancorfield Phil was always the one pushing gpg. No one used to bother until he bugged everyone 🙃 and the boot crowd never pushed it. #2018-11-05 16:15:13 dominicm @seancorfield do you mean exclusively with lein it broke, or it's broken altogether? #2018-11-05 16:15:56 dominicm @slipset I was thinking more about prompting if both commands gpg and gpg2 are available. Things like that. #2018-11-05 16:16:29 dominicm No rush. I think it's good to think on & compare. #2018-11-05 16:16:34 dominicm Hammock time 🙂 #2018-11-05 16:16:43 slipset I’d be really happy if you could comment on my comment there so I have something to hammock on 🙂 #2018-11-05 16:17:01 dominicm Sorry, I've got my emails turned off 😄 #2018-11-05 16:17:20 slipset Nothing to be sorry about, it’s actually very smart if you want to get things done 🙂 #2018-11-05 16:17:51 dominicm We have a large number of (non-)errors in a production environment at the moment, only way I can stay sane #2018-11-05 16:38:18 rickmoynihan FWIW on mac with GPG this is what I use and it seems to work: https://gpgtools.org/ #2018-11-05 17:39:40 seancorfield Yeah, that's what I was using but it stopped working and uninstalling/reinstalling didn't fix it so I gave up. The whole thing seemed more hassle than it was worth -- given the only thing that seemed to care about it was Leiningen... #2018-11-05 18:47:52 dominicm I consider gpg an important part of my security system, I think its a shame more people don't use it with jars.
But I can't speak to how well the gnu toolchain works elsewhere. #2018-11-05 19:10:16 slipset I used gpg --encrypt .clojars_creds.edn to encrypt my creds. #2018-11-05 19:10:36 slipset which may or may not be what you were asking for, as this is a symmetric encryption. #2018-11-05 19:13:03 dominicm I'll see if I can get it working with my smart card. Looking at the code, I don't expect it will, as I don't have a gpg passphrase. #2018-11-05 20:34:26 slipset May I ask what kind of smart card you have? #2018-11-05 21:16:49 dominicm Yubikey #2018-11-06 16:42:09 timgilbert Say, are there options available for customizing the groupId, version, etc in pom.xml during the course of clojure -Spom? #2018-11-06 16:42:41 dominicm @timgilbert update the generated pom.xml, then -Spom won't override them. #2018-11-06 16:43:59 timgilbert Hrm. I don't love having half-generated code in my codebase that I'll need to create another git commit for whenever I update deps.edn #2018-11-06 16:46:10 timgilbert I suppose I could write a tool that would run -Spom and then update the relevant values. What I'm actually looking for is something that will create the version number based on git tags on my repo in the same way lein-v does #2018-11-06 17:01:36 dominicm I imagine you're into "write another tool" territory 🙂 #2018-11-06 17:04:07 arrdem If someone writes a manifest generator, let me know 😉 I’m on the hunt for one. #2018-11-06 17:21:17 seancorfield @arrdem I forked depstar and added basic manifest generation -- then decided I really didn't need it and removed it. #2018-11-06 17:21:56 ghadi heh #2018-11-06 17:23:41 arrdem yagni #2018-11-06 17:24:05 arrdem I’m somewhat surprised how far bare jars have gotten me so far - we’ll see. #2018-11-06 17:25:06 ghadi I'm surprised by all the git deps users wanting to have artifacts back #2018-11-06 17:25:07 seancorfield We build/deploy bare uber JARs and run them that way in production. No AOT. java -cp path/to/the.jar clojure.main -m entry.point #2018-11-06 17:34:16 arrdem At this point I’m convinced that between Katamari / git deps I’ll probably only use external libraries from Maven again. Which is great. But I have internal users who aren’t on that workflow yet and who I want to be able to deploy artifacts with manifests and poms for. #2018-11-06 17:35:00 ghadi I get it from a transition strategy or reach perspective #2018-11-06 17:35:02 arrdem making nicer / faster psuedo-uberjars in docker is easy enough, doesn’t require any of this and that’s my only deployment need. #2018-11-06 19:02:21 orestis Can someone share some knowledge about the difference between bare jars and Uberjars and what manifests are for? #2018-11-06 19:02:35 orestis (Links welcome, no need to type it out here) #2018-11-06 19:02:47 orestis I finally got that Pom.xml is a maven thing :) #2018-11-06 19:07:53 orestis I’d love to be able to just rsync individual jars instead of an Uber jar, then somehow launch my app with the correct classpath. I guess if you have clj command line on the server installed, you can do that but you pay for the startup compilation time. #2018-11-06 19:08:49 dominicm @orestis Check out pack's skinny jar mode 🙂 #2018-11-06 19:15:04 seancorfield @orestis FYI, we deploy source uberjars to production -- no AOT -- and with all their dependencies included, they're still only 20-30MB so it's a pretty small payload to upload across servers (but, yes, would take a while to upload from a dev machine I guess). #2018-11-06 19:21:57 orestis I tried AOT jars and they still are roughly 30-40MB so still pretty small. It’s the compiling that takes time :) #2018-11-06 19:23:43 orestis And it’s annoying that you compile the same dependencies over and over. I’m fairly sure I’ve seen Alex & co discussing this as a potential improvement in the future. #2018-11-07 10:38:37 rickmoynihan Yes, I saw this the other day on the clojure mailinglist and was excited by it too.
It’s a mild annoyance that uberjaring and AOTing a moderately sized app takes 2 minutes. I understand why @U04V70XH6 likes to avoid AOTing apps; I’m assuming to have quicker builds… and I’ve been contemplating only AOTing nightlies, to bring CI times down… but still have uberjars start quickly.
I think this problem could be solved if we could easily AOT libraries. I can’t recall what the issues are here; I suspect it’s to do with how clojure.core/compilation/read/eval get linked in to libraries. #2018-11-07 15:33:09 seancorfield One big problem with AOT for libraries is that they can locked into specific versions of Clojure (this was a problem back in the day when some did compile their libraries for distribution). #2018-11-07 15:33:54 seancorfield With source-only libraries they should always work with all future versions of Clojure. #2018-11-07 15:37:37 seancorfield AOTing an application is safe because it's an entire system. Self-contained. For us, the startup time hit just isn't worth adding an AOT step. New Relic's instrumentation at startup is a much bigger overhead for us than Clojure's compilation. #2018-11-07 16:35:52 orestis Oh, can you share that new relic thing please? #2018-11-07 16:45:34 seancorfield @orestis What do you mean? We just use the regular Java agent for New Relic. I think we had to blacklist a particular classloader in Clojure to avoid prohibitively slow NR startup, but otherwise you just add the JVM startup options to point at their library and config file. #2018-11-07 16:47:00 seancorfield Happy to answer any other specific Qs you might have. #2018-11-07 16:50:10 orestis Right, that! Thanks so much! #2018-11-07 03:25:51 seancorfield Deleted the last vestiges of Boot from our system today (on dev). I decided to even convert two legacy system pieces since they are going to be around for a few months and it seemed silly to keep Boot around just for those. Very pleased with nice, simple, lean processes we have now. And we're on Clojure 1.10 Beta 5 in dev as well. Several blog posts to come 😉 #2018-11-10 04:18:18 jwhitlark Those blog posts are eagerly awaited. Do you think tools.deps has removed the need for boot, in general? #2018-11-10 05:16:09 seancorfield I think it depends what you want from a build tool. #2018-11-07 07:57:06 cfleming I’ve talked about some variant of this in the past, but I can’t remember the exact details now. It seems to be impossible to use tools.deps as a library from a process which doesn’t have the CWD set to the directory containing the main deps.edn file. This seems like a significant limitation - it means you can never have a process which resolves deps from two separate deps.edn files, for example. #2018-11-07 07:58:54 cfleming The problem is that if your deps.edn file references another using :local/root, the paths are incorrect - the paths from the dependency are relative to the CWD, not to the deps.edn file location. #2018-11-07 08:02:14 cfleming Here, root is the local :deps/root, which is generally relative. #2018-11-07 08:02:31 cfleming Is this intentional? If not, I’ll file a bug. #2018-11-07 08:06:34 cfleming I was somewhat convinced that this wasn’t previously the case, but it seems to have been this way for some time according to git. #2018-11-07 08:07:20 dominicm Always been this way, yeah #2018-11-07 08:13:52 cfleming @dominicm Is there an issue for it? There’s TDEPS-74 but that’s the transitive one. I can’t see anything else likely-looking. #2018-11-07 08:22:34 mpenet I think that also might be one of the reason katamari has vendored/forked deps for its use #2018-11-07 08:24:11 cfleming Interesting, I’ll have to check that out. #2018-11-07 08:28:52 cfleming Right, looks like it uses raynes/fs #2018-11-07 08:30:52 cfleming This is kind of a show-stopper for Cursive, unfortunately. #2018-11-07 08:31:16 dominicm Uncertain. I think there should be. #2018-11-07 17:08:42 orestis What about CLJ_CONFIG? I remember @seancorfield uses it for something similar? #2018-11-07 17:18:15 dominicm @orestis that's for having multiple deps.edn files. It's bad for tooling as it stops users from adding personal aliases to use. #2018-11-07 17:24:22 seancorfield Well, it's good for projects that want a specific deps.edn "base" file instead of using whatever the developer happens to have in their .clojure/deps.edn file -- so it really depends on your perspective. #2018-11-07 17:27:37 seancorfield For our monorepo, we specify all our defaults (some aliases, but mostly :override-deps) in a versions/deps.edn file and we use CLJ_CONFIG to use that as our basis for all the subprojects' deps.edn files. #2018-11-07 17:28:17 seancorfield That doesn't help Cursive tho', since it affects the "wrong" deps.edn file -- the "user" one, not the "project" one. #2018-11-07 17:29:50 seancorfield In one place, we shell out to the clojure command and when we do that (via clojure.java.shell/sh) we specify the :dir that it should be executed in so CWD is controlled. I'm not sure how Cursive invokes this stuff -- I suspect it is trying to use t.d.a. directly and that's where the problem occurs...? #2018-11-07 17:33:23 dominicm cursive is java, so it's using it as a library rather than shelling out. I think @cfleming mentioned that $PATH isn't set in cursive. #2018-11-07 17:38:32 seancorfield In our Clojure code, we still shell out to run clojure 🙂 #2018-11-07 18:53:36 cfleming What I want is to get the full Java command line that I need to execute, so that I can execute it with IntelliJ’s command running infrastructure. There’s no way to get that from the command line with clj. #2018-11-07 18:58:44 cfleming @dominicm The issue is that if you use t.d.a as a lib, there’s no way to set the CWD of a running Java process. It’s not a $PATH thing. #2018-11-07 19:36:45 mpenet I guess it's jni under the hood #2018-11-07 20:29:31 cfleming Yeah, but that would probably break all sorts of things with IntelliJ (where the logs go, etc). #2018-11-07 19:19:42 dominicm Duh, of course #2018-11-07 19:32:50 rickmoynihan @cfleming: is an entirely insufficient workaround to launch idea from the command line in your projects working directory? #2018-11-07 19:34:33 rickmoynihan IIRC I’ve done that in the past with intellij; when I’ve run into issues with macos/quartz-UI not being a child of a shell environment #2018-11-07 19:35:22 rickmoynihan though I don’t normally use intellij #2018-11-07 20:10:46 arrdem @rickmoynihan yeah - most IDEA launches are through a GUI, and that defeats IDEA’s multi-project/workspace capabilities. #2018-11-07 20:13:30 rickmoynihan … but it does work around the issue? #2018-11-07 20:27:54 cfleming @rickmoynihan Only if you ever use one project - I usually have 3 open. #2018-11-07 20:30:41 rickmoynihan as I said an entirely insufficient workaround #2018-11-07 20:28:43 cfleming I actually don’t even know how to run a Mac app from a specified directory, or if it’s even possible. #2018-11-07 20:30:06 rickmoynihan /Applications/IntelliJ\ does it for me #2018-11-07 20:30:40 cfleming For now, I’ll probably just have to do the same horrible hacks I do for leiningen using hooke or the moral equivalent. #2018-11-07 20:32:29 cfleming @arrdem Has raynes/fs worked well for your vendored fork? #2018-11-07 20:32:49 cfleming Looking at the code, I assume this sort of problem is what you’re trying to deal with? #2018-11-07 20:32:51 arrdem @cfleming Yeah. It was a bit fiddly to figure out where I needed it, but I haven’t had problems with it since. #2018-11-07 20:33:06 arrdem You’re right, that is why I patched it in. #2018-11-07 20:33:28 cfleming Are there places where you don’t want it? If I use hooke, then that’s pretty global. #2018-11-07 20:34:05 arrdem Can you hooke or alter-var-root on io/file? That seems like a really big hammer and may play badly with static linking. #2018-11-07 20:35:14 arrdem In general I couldn’t prove that things wouldn’t behave precisely as expected and erred on the side of caution, so I did the refactor by hand and it took a couple passes. #2018-11-07 21:47:31 cfleming (sorry, got called away there) #2018-11-07 21:47:56 cfleming Yeah, that’s what I do. It’s scoped to a particular sequence of calls, but during that sequence it’s globally modified, yes. #2018-11-07 21:48:25 cfleming It seems to work, and I think it works because the code calling io/file is not statically linked, so it uses the var. #2018-11-07 21:48:56 cfleming @arrdem ^^ #2018-11-07 21:49:16 arrdem hummm #2018-11-07 21:49:18 arrdem exciting #2018-11-07 21:49:36 cfleming I’m not sure whether that’s better or worse than vendoring a specific version of deps, which is then tied to that version of Cursive. #2018-11-07 21:50:01 arrdem For katamari it’s as much a proof that katamari’s heavily vendored monorepo workflow actually does work. #2018-11-07 21:50:03 cfleming All options suck, basically. #2018-11-07 21:50:11 arrdem fork && vendor && move on #2018-11-07 21:51:33 cfleming I can’t wait for the bug reports saying that Cursive xxx doesn’t support the functionality that was introduced in deps yyy #2018-11-07 21:52:14 arrdem yeah I need to write some bash + sed automation to let me packport merged diffs ~automatically. #2018-11-07 22:12:54 cfleming The problem I have with that is that I can’t rely on everyone upgrading Cursive, so older versions will still be out there with older versions of tools.deps baked in. #2018-11-08 13:21:44 onetom btw, we are relying more and more on project specific nix-shells (activated via [`direnv`](https://direnv.net) by simply cding into a project's directory)
and intellij doesn't expects such approach at all, instead it's providing tons of custom dialogs with very similar path and option settings for the various tools in different plugins 😞
i really envy emacs in this respect, since they have https://github.com/wbolster/emacs-direnv #2018-11-08 15:56:14 richiardiandrea Definitely nice will try it out #2018-11-08 13:59:52 dominicm Neat, it supports vim too 🙂 #2018-11-11 07:50:26 jwhitlark #2018-11-11 08:18:48 dominicm It is normal behaviour from my past testing on this. The output directory is supposed to already be in the classpath. #2018-11-11 08:31:32 jwhitlark Ok. #2018-11-11 08:53:03 orestis Does tools-deps have a story for compiling Java code that lives in the same repo? #2018-11-11 12:43:38 danieroux Joined to ask this exact question. I have one measly .java file that I want to (:import), and i really don't want to make a jar just for that one file. #2018-11-11 08:53:20 orestis (No immediate use case, just curious) #2018-11-11 09:02:08 jwhitlark I'm curious about that too. #2018-11-11 09:24:33 dominicm Not at all @orestis. #2018-11-11 09:25:24 dominicm I've done some work on this, but it would be the same level as clojure.core/compile I suspect. Riddler is a good option to look at I think #2018-11-11 12:46:33 danieroux what is Riddler, where can I find it? #2018-11-11 13:02:33 dottedmag tools.deps is about building classpath, clj is about running Clojure. You can run javac from one of clj aliases if that's a single .java file. #2018-11-11 23:17:16 jwhitlark Wow, digging through virgil's code answers a lot of questions. #2018-11-12 00:54:24 gfredericks there's no way to add a jvm-opt without having to enable an alias via CLI opts, is that right? #2018-11-12 01:34:54 mfikes @gfredericks -J #2018-11-12 01:35:38 gfredericks @mfikes sorry, I mean I'd like to put it in the deps.edn #2018-11-12 01:35:41 mfikes Example: clj -J-Xmx512m #2018-11-12 01:35:44 gfredericks and have, e.g., clojure pick it up #2018-11-12 01:35:50 gfredericks without any extra args #2018-11-12 01:36:37 mfikes Ahh, I think you are right in that case. I can't think of a way to just "make it so" #2018-11-12 01:38:37 mfikes It would seem, a top-level thing, sibling of :deps or :paths is where a :jvm-opts could live #2018-11-12 01:39:04 gfredericks yeah; I can't tell if that is just a missing feature, or if there's some philosophical conflict #2018-11-12 01:39:58 alexmiller It’s missing, no philosophical objection #2018-11-12 19:11:14 adamfrey here's a command that I can run from the terminal to start a cider-ready nrepl, but I'm having a problem when putting it into an aliase in a deps.edn file.
clj -Sdeps '{:deps {nrepl {:mvn/version "0.4.5"} cider/cider-nrepl {:mvn/version "0.18.0"} refactor-nrepl {:mvn/version "2.4.0"}}}' -m nrepl.cmdline --middleware '[cider.nrepl/cider-middleware refactor-nrepl.middleware/wrap-refactor]'
the --middleware option is a vector with a space in the middle, but I can't find any way to make it work from deps.edn.
:main-opts ["-m" "nrepl.cmdline"
"--middleware" "[cider.nrepl/cider-middleware refactor-nrepl.middleware/wrap-refactor]"
"--interactive"]
by the time it gets to the -main function the args binding is set to
("--middleware" "[cider.nrepl/cider-middleware" "refactor-nrepl.middleware/wrap-refactor]" "--interactive")
#2018-11-12 19:36:08 dottedmag @adamfrey try , instead of a space #2018-11-12 19:39:40 adamfrey oh good call! it works because it gets passed as one thing to the nrepl main and then nrepl parses it from a string using edn/read-string Thanks! #2018-11-12 21:53:42 martinklepsch Also known as the Corfield Comma 😂 #2018-11-12 20:22:28 seancorfield Answered on ClojureVerse. #2018-11-12 20:22:56 avi Thanks Sean! 🙏 #2018-11-12 20:23:05 seancorfield TL:DR; dependency version resolution has a different set of versions to consider with one alias vs multiple aliases. #2018-11-12 20:24:05 avi I think I get it; makes sense 😅 #2018-11-15 16:01:19 avi Just wondering if anyone’s figured out a way to get tools.deps to use a http proxy when downloading Maven deps, other than applying the patch in https://dev.clojure.org/jira/browse/TDEPS-20 ? I’m just being lazy I guess, putting off learning how to patch tools.deps and install and use the patched version… #2018-11-15 16:07:27 alexmiller I don’t think there is any way to do it but to fix that (and would be happy to see a patch for it) #2018-11-15 16:07:41 alexmiller oh, there is a patch for it, sorry #2018-11-15 16:07:51 alexmiller well, then you just have to wait :) #2018-11-15 16:08:03 alexmiller I’m probably not going to have much time to look at tools.deps stuff until post-conj #2018-11-15 16:54:19 avi totally understandable, no worries! Thanks for the clear and helpful response! #2018-11-15 17:34:55 martinklepsch Is there any documentation on how tools.deps resolves deps? Specifically wondering what happens when two dependencies depend on 'A but in different versions #2018-11-15 17:35:16 alexmiller the general idea is to always take the newest #2018-11-15 17:36:14 martinklepsch Ok, that's great. Does that "general idea" imply any exceptions to that rule where the older artifact would be chosen? #2018-11-15 17:36:59 alexmiller the complexities come when you consider the full graph #2018-11-15 17:37:22 alexmiller A -> B1 -> C2
-> B2 -> C1 #2018-11-15 17:37:52 alexmiller do you get {A, B1, C2} or {A, B2, C1} or {A, B2, C2} #2018-11-15 17:38:25 timgilbert I'm glad I'm not the one who needs to figure that out 😉 #2018-11-15 17:39:01 alexmiller it’s actually NP-complete I believe #2018-11-15 17:39:14 alexmiller in the general case #2018-11-15 17:42:14 alexmiller fortunately most actual cases are pretty “easy” #2018-11-15 17:52:52 timgilbert ...unless they involve jackson, I'm guessing #2018-11-15 18:00:06 alexmiller yes, that is the exception :) #2018-11-15 18:00:26 alexmiller and unfortunately, every dependency set includes at least 9 versions of jackson #2018-11-15 18:56:20 andy.fingerhut A new complexity class: jackson-complete #2018-11-15 18:56:59 andy.fingerhut Although that might be equivalent to: undecidable 🙂 #2018-11-15 18:58:52 mfikes Entjaksonsproblem #2018-11-15 19:06:23 andy.fingerhut Oh, that is good. You even Deutsch-ified the spelling of Jackson. Love the attention to detail there 🙂 #2018-11-15 19:13:28 alexmiller now that’s a t-shirt with sufficiently small audience #2018-11-15 19:16:02 andy.fingerhut A sufficiently small audience to .... not make it? #2018-11-15 19:25:46 alexmiller to be fun :) #2018-11-15 19:27:06 alexmiller a joke t-shirt with an audience of 1 is not fun, because you can’t share it. And a joke t-shirt with an audience of large N, is cliche. So you have to find that sweet spot in the middle. :) #2018-11-15 19:27:53 alexmiller this is bigger than 1, but not by much :) #2018-11-15 19:29:48 andy.fingerhut Got it. It is pretty easily explained to anyone at the Conj, if they don't get it immediately (I wouldn't without this conversation) #2018-11-15 19:32:44 andy.fingerhut As a reminder, if you think Rich would see a "channeling Rich sayings" T-shirt as a tribute (as it is intended), not a weirdness or offensive, I would be happy to bring some to the Conj 🙂 #2018-11-15 19:39:40 alexmiller prob not the best idea #2018-11-15 19:39:48 andy.fingerhut But also happy to never bring it up again if he doesn't want to see such things. #2018-11-15 19:39:59 alexmiller for someone with such great care for words, I hate to ascribe any words to him #2018-11-15 19:41:27 andy.fingerhut Understood. He isn't the only person like that, and such people should get to be upset with people that misquote them, especially if the meaning is changed. FYI, I do take pretty significant care in transcribing his talks. I think I've listened to the ones I have transcribed at least 4 times each now. #2018-11-15 19:41:46 seancorfield Maybe we should make some (rich (comment forms))
T shirts? off-topic #2018-11-15 19:42:04 alexmiller I don’t get that #2018-11-15 19:42:22 alexmiller I appreciate the care in the transcripts - he’s never complained about those #2018-11-15 19:42:47 seancorfield Stu talked about Rich Comment Forms in his "Running With Scissors" talk. #2018-11-15 19:42:48 alexmiller and I know he appreciates having slides+text together for context #2018-11-15 19:42:57 alexmiller oh, I haven’t watched it yet :) #2018-11-15 19:43:29 seancorfield And observing it was a pun on the (comment ...) forms being "rich" with exploratory code and it also being a habit of Rich to use comment that way. #2018-11-15 19:44:08 alexmiller clearly the circle is too small for a t-shirt :) #2018-11-15 19:44:49 alexmiller fwiw, I made a custom “O(log32 n) is good enough” t-shirt for Conj 1 :) #2018-11-15 19:54:05 dpsutton if you donated that design to clojurist together i bet they could raise some money for projects by selling the t shirt. i would for sure get one #2018-11-15 20:06:17 alexmiller well feel free to make one yourself, mine was certainly nothing special and I don’t have it #2018-11-15 20:09:31 andy.fingerhut http://customink.com makes it really easy to create T-shirt designs, I found out recently #2018-11-15 20:10:25 andy.fingerhut I have never bought anything from them, so don't have knowledge of the full experience front to back. #2018-11-15 20:13:37 dpsutton i made some tshirts for my team at my last company through them. I still get emails to make mouse pads and other things with that design 2.5 years later #2018-11-15 20:13:42 dpsutton but the shirts were/are great #2018-11-16 01:28:20 gfredericks ...mouse...pads? #2018-11-16 01:28:53 gfredericks I don't think I've even seen a mause pad for at least fifteen years #2018-11-15 19:46:12 alexmiller well, all the conference t-shirts too :) #2018-11-15 20:18:17 cfleming I still love my conj 2014 t-shirt. #2018-11-15 20:18:48 dpsutton my clojure west 2016 and strangeloop 2016 are my two favorite shirts #2018-11-15 20:18:50 cfleming Not many other conference t-shirts stick around for long. #2018-11-15 20:19:31 andy.fingerhut And here I thought the biggest benefit of the conferences was listening to the talks, and the hallway track 🙂 #2018-11-15 20:20:06 cfleming No, listening to the talks is totally overrated, it’s only the hallway track 🙂 #2018-11-15 20:20:56 cfleming At least for me, since I can watch talks online, but seldom get to chat with others who understand what I’m talking about. #2018-11-15 20:21:19 andy.fingerhut But clearly I have missed out on some great T-shirt buying opportunities that are not available elsewhere. This should be in the conference advertisements! #2018-11-16 09:05:09 martinklepsch has there been any discussion around preventing things like this:
(tdeps/resolve-deps
{:deps '{clj-time {:mvn/version "0.8.0"}
clj-time/clj-time {:mvn/version "0.7.0"}}
:mvn/repos default-repos}
nil)
#2018-11-16 13:14:58 alexmiller Should be fine - there is a canonicalization step #2018-11-16 17:51:32 tcrawley I'm working on migrating a project from lein to tools.deps, and need to access an authenticated repo. https://clojure.org/reference/deps_and_cli#_procurers says that the credentials are pulled from ~/.m2/settings.xml, which doesn't work well for us. Is there another method for specifying credentials, or plans to expose one? #2018-11-16 17:51:53 alexmiller in short, no #2018-11-16 17:52:07 tcrawley I'd be fine with embedding the credentials in deps.edn - ours are generated by make, and are never checked in #2018-11-16 17:52:27 alexmiller we are not fine with putting any kind of credentials in deps.edn so will not be doing that #2018-11-16 17:53:40 alexmiller perhaps there is some other way to pull credentials from a trusted source #2018-11-16 17:55:31 tcrawley What about a tools.deps-specific file for them? ~/.tools-deps.edn would possibly work, and is as secure as ~/.m2/settings.xml #2018-11-16 17:56:13 dominicm @tcrawley settings.xml is more secure. It supports encryption. Unless tools.deps adds encryption. #2018-11-16 17:56:16 alexmiller why is that different for you from making a ~/.m2/settings.xml? #2018-11-16 17:56:35 alexmiller have no desire to do the encryption stuff #2018-11-16 17:56:54 tcrawley @dominicm does tools.deps support encrypted ~/.m2/settings.xml? #2018-11-16 17:57:08 alexmiller it uses the Maven apis, so should #2018-11-16 17:57:20 tcrawley ah, that makes sense #2018-11-16 17:57:33 alexmiller can’t say that I’ve actually done it #2018-11-16 18:00:03 dominicm @tcrawley if you wanted to get really crazy. You could generate a server into ~/.m2/settings.xml and use a reference to that in deps.edn. #2018-11-16 18:00:13 dominicm xml is just data after all. #2018-11-16 18:01:35 tcrawley right, I'd just like to avoid that if possible :) I can output an alternate settings.xml, then tell tools.deps to use that if the maven libs support setting that via an environment variable and it actually works #2018-11-16 18:01:38 tcrawley I'll try that first #2018-11-16 18:05:05 dpsutton there's also a bug with the credentials in settings.xml. its on github issues for s3 wagon maybe? alex had very useful information in it #2018-11-16 18:05:16 dpsutton it affects my ubuntu coworkers but not me on fedora right now #2018-11-16 18:18:43 alexmiller that is not an issue with tools.deps (as I am using a different version than s3-wagon-private) #2018-11-16 18:29:00 kenny Why does this produce a REPL instead of running my main opts?
clj -Sdeps '{:aliases {:jar {:extra-deps #:com.healthfinch{depstar {:git/url "", :sha "2879672df7fd296855563c22dd9ba9de2022a0b4"}}}, :main-opts ["-m" "hf.depstar.uberjar" "foo.jar"]}}' -A:jar
Clojure 1.9.0
user=>
#2018-11-16 18:29:50 ghadi because the ns'ed map syntax is qualifying :sha incorrectly #2018-11-16 18:29:51 kenny Wow, :main-opts is at the wrong level. Oops... #2018-11-16 18:30:04 kenny Great start to the day 🙃 #2018-11-16 18:30:10 ghadi main opts isn't supported top-level, AFAIK #2018-11-16 18:30:36 ghadi but yeah it's at the wrong level #2018-11-16 18:31:42 kenny Shouldn't there be some validation there? #2018-11-16 18:31:56 kenny I passed a vector instead of a map to an alias. #2018-11-16 21:28:32 tcrawley I figured out how to tell the maven lib to use a different settings.xml file by setting the org.apache.maven.user-settings sysprop. However, clojure doesn't expose a way to pass options to the make-classpath java invocation. Is that something you would be interested in exposing @alexmiller? #2018-11-16 21:29:00 tcrawley I can work around it with a custom copy of the clojure script for now #2018-11-16 21:31:04 alexmiller Would prob rather add a top level :mvn/ key but would be good to have a ticket regardless #2018-11-16 21:32:39 tcrawley I was thinking about a general way to pass options to that invocation, but a top-level :mvn/settings-file (or whatnot) would work for this use-case. I'll file a ticket. #2018-11-16 21:40:54 dominicm I initially thought it would not work, but as long as -Sdeps is considered, it works quite well. #2018-11-16 21:43:51 tcrawley @dominicm what "it" are you referring to? #2018-11-16 21:44:17 dominicm A top level key rather than an environment variable @tcrawley #2018-11-16 21:44:39 tcrawley gotcha #2018-11-17 18:26:34 richiardiandrea I am trying to generate a pom with version and project coords coming from the outside #2018-11-17 18:26:38 alexmiller No, but it syncs - so you can change it and it won’t be removed #2018-11-17 18:28:30 richiardiandrea Yep I saw the sync but in my case the info is coming from another file (either project.clj or package.json) and I think it would be awesome to have the pom.xml generated somehow - not persisted. Is it something worth discussing about? #2018-11-17 18:32:07 richiardiandrea This is the only missing piece for an automated - deps.edn + project descriptor - no manual steps flow 😃
I know there exists probably a maven plugin for bumping versions in a pom but ...it is no very lean #2018-11-17 18:33:33 richiardiandrea And thanks for the quick answer as well #2018-11-17 18:37:16 richiardiandrea Or I can write some bash but when this happens it always feels there is something hacky I don't know why simple_smile #2018-11-18 00:20:52 ro6 as tools.deps stands today, if I'm co-developing an app+library locally, do I need to cycle my clj repl to refresh changes from the library? #2018-11-18 00:40:17 alexmiller Depends but you can probably just reload the namespace #2018-11-18 06:21:44 dominicm @robert.mather.rmm tools.namespace refresh will reload namespaces in local libraries. #2018-11-19 19:06:05 martinklepsch This fails to find org.jboss.naming:jnpserver:jar:5.0.3.GA. org.immutant/messaging has the jboss repo specified in it's pom but only transitively depends on jnpserver. The the immediate dependencies of org.immutant/messaging do not specify the jboss repository.
clj -Sdeps '{:deps {org.immutant/messaging {:mvn/version "2.1.10"}}}' -Stree
#2018-11-19 19:06:34 martinklepsch Is this something worth opening a JIRA ticket for? #2018-11-19 19:10:03 alexmiller I think there is one? #2018-11-19 19:12:54 alexmiller Well that’s for -Spom right? Related but different #2018-11-19 19:13:37 alexmiller Well no, guess not #2018-11-19 19:13:49 alexmiller I find the way that ticket is written confusing #2018-11-19 19:13:53 martinklepsch yeah... but there's a ticket for -Spom as well #2018-11-19 19:27:35 martinklepsch @alexmiller btw, I checked up on canonicalize-deps and either it's not working or I'm misunderstanding what it's supposed to do:
(let [cfg {:deps '{clj-time {:mvn/version "0.8.0"}
clj-time/clj-time {:mvn/version "0.7.0"}}
:mvn/repos mvn/standard-repos}]
(canonicalize-deps (:deps cfg) cfg))
;; => [[clj-time #:mvn{:version "0.8.0"}]
;; [clj-time/clj-time #:mvn{:version "0.7.0"}]]
#2018-11-19 19:35:55 alexmiller there’s actually two different points where things get canonicalized #2018-11-19 19:36:30 alexmiller the other is one of the extension methods that gets called by resolve-deps, and I think that’s where this gets adjusted #2018-11-19 19:37:00 alexmiller I started doing this earlier in canonicalize-deps and then ended up incorporating it later in the process instead #2018-11-19 19:37:27 alexmiller sorry, don’t have time to look at it now #2018-11-19 20:49:19 borkdude In the case of git deps, which paths of the repo get added to the classpath? Does it look at a pom file? Can you control this? E.g. also add the test dir so you can run the tests of a git dep? #2018-11-19 21:43:40 alexmiller git deps will detect the manifest type (deps.edn or pom.xml) and use the paths indicated #2018-11-19 21:46:07 alexmiller you can override it with :deps/manifest :deps or :deps/manifest :pom #2018-11-19 21:48:04 borkdude I guess that’s only useful if both are present, else it will choose whichever is present? #2018-11-19 21:48:15 alexmiller prefers deps.edn #2018-11-19 21:48:17 alexmiller then pom #2018-11-19 21:48:32 alexmiller if neither will throw, but if you explicitly set to :deps, it is tolerant of a missing deps.edn #2018-11-19 21:48:41 alexmiller and just treats it as a dep with no transitive deps #2018-11-19 21:49:02 borkdude in that case it chooses ["src"] as the default :paths? #2018-11-22 08:30:48 mkvlr I’m trying to get add-lib working with prepl. I’m using the latest add-lib branch and clojure: #2018-11-22 08:30:55 mkvlr clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-beta7"} org.clojure/tools.deps.alpha {:git/url "" :sha "f6c080bd0049211021ea59e516d1785b08302515"}}}' -J-Dclojure.server.repl="{:port 5556,:accept clojure.core.server/io-prepl}" #2018-11-22 08:32:04 mkvlr to get it working I have to set the dynamic class loader first: #2018-11-22 08:32:11 mkvlr (let [cl (.getContextClassLoader (Thread/currentThread))
dcl (clojure.lang.DynamicClassLoader. cl)]
(.setContextClassLoader (Thread/currentThread) dcl))
#2018-11-22 08:40:28 mkvlr or is adding functionality like this the job of the tool using prepl? #2018-11-22 10:19:16 Ben Hammond Hi guys I'm working on a project that extends
https://github.com/juxt/edge
which daisy-chains a bunch of small dep.edn files together using
{:local/root "../edge.kick"}
I'm trying to get my head round how this plays with alias declarations
⚓ #2018-11-22 10:20:51 Ben Hammond so I've just discovered that it doesn't
aliases are only sampled at the root level #2018-11-22 10:22:01 Ben Hammond which implies to me that I have misunderstood the intention of :local/root #2018-11-22 10:22:52 Ben Hammond is there a description of what :local/root should, and should not be used for #2018-11-22 10:22:58 Ben Hammond that you can point me at #2018-11-22 10:23:51 Ben Hammond I guess I'm trying to strong-arm deps into working like an OSGI style module system #2018-11-22 10:24:10 Ben Hammond and I'm wondering if that would be considered abusive #2018-11-22 10:39:51 rickmoynihan I’ve had similar ideas about module systems.
In particular I have a nagging feeling that integrant/duct + deps.edn could be such a thing. #2018-11-22 10:44:17 Ben Hammond yes you are probably right #2018-11-22 10:44:49 Ben Hammond I think my confusion is this #2018-11-22 10:45:45 Ben Hammond :local/root dependency allows you to see the internal development gubbbins of a dependency #2018-11-22 10:46:18 Ben Hammond which creates the temptation that you can manipulate those other aliases #2018-11-22 10:47:08 Ben Hammond but at some point you should expect to move that dependence to a fixed binary #2018-11-22 10:48:00 Ben Hammond therefore the thought process should always be How would I use this if it was a binary artefact #2018-11-22 16:40:16 mfikes The fact that :local/root behaves just like :git/url or :mvn/version with respect to aliases is a good thing #2018-11-23 12:39:16 prepor Hello. I want to suggest another way how to build uberjars with tools.deps. boot-tools-deps amends boot's configurations. Our approach doesn't complement uber task but replaces it.
https://gist.github.com/prepor/ad40e061287793660c9faff21de23b82
This task simply adds classpaths (and paths) from tools.deps to fileset. And it works with git and local dependencies too. The final task could look like this:
(deftask build
[]
(comp (deps-uber)
(aot :namespace '[app.core])
(jar :file "standalone.jar"
:main 'app.core)
(sift :include #{#"standalone.jar"})
(target)))
#2018-11-23 19:00:40 dominicm @alexmiller https://dev.clojure.org/jira/browse/TDEPS-17 has become a priority for me. I can't use our git fork because it's coming in transitively.
I'd like to work on it this weekend, any thoughts you've formed so far? I'm thinking that the simplest version is that something like this will work:
org.clojure/tools.namespace {:mvn/version "1.0.0" :lib pro.juxt/tools.namespace}
Version is first priority. 2 with equal versions and different libs throws an error like comparing git to mvn does. #2018-11-24 21:04:24 jwhitlark I haven't seen a bash/zsh completion script for clj, but it occurs to me that a) it wouldn't be that hard, b) the project specific part could use direnv to set that portion when you entered a clojure project directory. Any input? #2018-11-25 10:26:34 dominicm I have an interesting conundrum in pack. I'm trying to suppress this output:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/dominic/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
Suppressing it for my projects is easy enough, I add exclusions to pack for slf4j-nop.
However, on projects which don't already have their own logging stack, it then results in:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Which isn't a better result imo.
The ideal is that the -main function's in pack would tell slf4j what to use if there's multiple bindings, suppressing the warning. But I don't see an obvious API for that. Anyone else got any good ideas? #2018-11-25 12:55:00 borkdude Common mistake of mine:
$ clj -Srepro -Sdeps '{:deps {org.clojure/test.check {:mvn/release "RELEASE"}}}'
Error building classpath. nil
java.lang.NullPointerException
Maybe an error message could help here? #2018-11-26 17:41:24 rickmoynihan Does anyone know if this issue likely to be resolved?
https://dev.clojure.org/jira/browse/TDEPS-82
I think it causes lein-tools-deps (which first uses -Sdescribe) to resolve dependencies unnecessarily and too soon #2018-11-26 18:50:51 alexmiller Yes, would like to fix #2018-11-26 20:01:22 seancorfield I patched the clojure script locally and was hoping to provide a patch to that ticket but my weekends have not been conducive to that, unfortunately. #2018-11-26 20:01:51 seancorfield (I'm not sure my quick'n'dirty local patch is an ideal solution to submit yet) #2018-11-26 20:27:17 alexmiller yeah, sorry I haven’t had a chance to look at it yet #2018-11-26 20:27:29 alexmiller I’m hoping to have some cycles post-conj to catch up on tools.deps stuff #2018-11-27 15:45:45 adamfrey can :jvm-opts be put at the top level of a project's deps.edn file?
like:
:jvm-opts ["-Xms512M"
"-Xmx1g"
"-Djava.io.tmpdir=./tmp"
"-Dcatalina.base=./tmp"]
#2018-11-27 15:46:14 adamfrey It doesn't seem to be working but I can't find an example in the docs so maybe that's not a supported feature #2018-11-27 16:00:32 dominicm I guess it doesn't make much sense for that to exist does it? #2018-11-27 16:03:25 adamfrey why doesn't it make sense? I would like to start a a clojure process from my project and have a bunch of jvm opts applied without having to specify them in the cli each time. #2018-11-27 16:10:16 dominicm because it works from an alias #2018-11-27 16:15:18 dominicm I'm thinking about the expectations I have, which is that :aliases is ignored from deps.edn transitively, and everything else is transitive. #2018-11-27 20:39:58 aisamu I'd say it follows the same logic as a root-level :main-opts #2018-11-27 21:53:46 dominicm That doesn't exist either! #2018-11-27 22:04:06 aisamu Precisely simple_smile #2018-11-27 16:29:15 eval2020 Can one require deps in the init-script? #2018-11-27 16:31:39 eval2020 This succeeds: clj -e '(require (quote [clj-time.core :as t])) (prn (t/now))' but I cannot make it work from init-script. #2018-11-27 16:36:10 eval2020 I was doing weird stuff: yes this works as expected #2018-11-27 18:39:44 dominicm thought this was a need trick:
#!/bin/sh
deps="{}"
if [ "$#" -gt 0 ]; then
deps="$(clojure -A:user/find-deps "
I named it replme. #2018-11-27 22:25:36 winkywooster @dominicm that’s cool, thanks for sharing #2018-11-28 10:56:57 eval2020 @dominicm tools-deps equivalent of lein-try - very nice! 👌:skin-tone-2: #2018-11-28 12:33:37 borkdude what does user/find-deps do? #2018-11-28 16:28:08 dominicm What's the :repl/try about? #2018-11-28 20:28:42 borkdude Is there a way to get -R test + -C test + also the possibility to override -m? With -A:test I cannot do this. #2018-11-28 20:30:47 eval2020 @dominicm it’s a hook from rebel that kickstarts the loading of the libs (calling (unreleased) tools-deps’ add-lib). (Must be an easier way to start a repl where you want some commands to be executed before taking input) #2018-11-28 21:54:46 dominicm There is, edge does it #2018-11-28 21:55:21 dominicm Rebel takes the same flags as clojure main #2018-11-28 22:26:51 kenny Is there a way to get rid of the logback.xml file that aws-maven brings in via tools-deps? #2018-11-28 22:29:14 dominicm @kenny I solved it by making a fork and opening a jira. #2018-11-28 22:31:54 kenny Unfortunately I need S3 support because Datomic Cloud stores its artifacts there 😞 #2018-11-28 22:34:02 alexmiller I am interested in fixing that for you, after conj (although there may be new ways to do so) #2018-11-29 06:09:57 dominicm That's great, thanks! #2018-11-28 22:34:39 dpsutton would make a fun unsession to fix a bug with alex miller #2018-11-28 22:37:09 alexmiller Oh yeah so fun ;) #2018-11-28 22:37:35 dpsutton "work during the conference. thanks" lol #2018-11-28 22:51:18 mattyulrich Hi all. I'm having an issue that looks like transitive dependency problems with a corporate repo. I have the repo listed in :mvn/repos - but it looks like transitive deps from java libraries that I reference from that repo are not resolving. When trying to resolve the deps, it seems to be looking at clojars instead. Is this a known issue or something that should be working? #2018-11-28 22:57:22 hiredman there are a lot of issues in the tools.deps jira, so in order to discern you are experiencing a known issue or not, you may have to narrow it down #2018-11-28 23:20:33 amar Hi everyone. Has anyone been able to use a private git library as a dependency in a CI environment like Google Cloud Build? The key is there and if I use git from the command line, the repo can be successfully cloned.
I'm seeing
com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available:
at com.jcraft.jsch.agentproxy.ConnectorFactory.createConnector(ConnectorFactory.java:120)
at clojure.tools.gitlibs.impl$fn__843.invokeStatic(impl.clj:30)
at clojure.tools.gitlibs.impl$fn__843.invoke(impl.clj:28)
at clojure.lang.Delay.deref(Delay.java:42)
#2018-11-29 06:10:51 dominicm You need to start an ssh agent, and add the key to it #2018-11-29 21:06:53 amar Great. Thanks! #2018-11-29 00:00:18 mattyulrich Thanks @hiredman - don't see a dependency relocation, but it does look like a good list of Jiras to review. I'll dig into there and see if I can see my issue. #2018-12-01 18:01:06 pesterhazy Unfortunately with clj -A:nrepl:in-ns, the -e (...) argument for the second alias overrides the -e (...) of the first #2018-12-01 18:01:42 pesterhazy Is this intended behavior? Is there another way for an alias to execute code? #2018-12-01 18:02:55 pesterhazy Note that clj -A:nrep -e (...) does work #2018-12-01 18:21:52 dominicm Non composition is expected. #2018-12-01 18:26:52 pesterhazy Those are two enhancements to clj that would make aliases more useful then: avoiding the need for Corfield Commas, and being able to stack side effects (like starting an nREPL server) on top of each other #2018-12-01 21:46:16 alexmiller You can stack effects with -m - main args are appended to the end. But, there are other things going on in your example above (no support for -e aliases, ordering of -A vs other aliases, etc) #2018-12-02 11:23:54 pesterhazy so in this case it's better to use -M:alias1:alias2 than -A:alias1:alias2? #2018-12-02 11:48:04 dominicm The former would have no meaning #2018-12-02 11:48:24 dominicm Well, the former is equivalent to -M:alias2 #2018-12-02 11:49:34 pesterhazy Ok, I'll give that a try #2018-12-02 11:50:32 dominicm I opened an nrepl issue to start a discussion about how this might work. I've not come up with anything good. #2018-12-02 11:58:58 pesterhazy @U09LZR36F can you link to the issue? #2018-12-02 12:05:35 dominicm I should update it to mention the actual problem statement #2018-12-02 12:05:55 dominicm I did think about a main Hopper #2018-12-02 12:06:47 dominicm A simple version could be put together in an afternoon #2018-12-02 14:27:24 alexmiller sorry, I meant you could stack effects from last alias + command line, not across aliases. I think making ordering do stuff beyond that is too fragile #2018-12-02 14:29:18 alexmiller at some point, you are putting too much on CLJ and should recognize that you are writing a program and push this complexity into a Clojure program I think #2018-12-02 14:29:34 alexmiller but I’m willing to consider that I’m wrong on that :) #2018-12-02 15:06:36 dominicm It's a little bit complicated, because I think mains have been pushed as a common interface. But they don't compose exactly.
But anything you want to do, can be done with jvm opts and a custom main. #2018-12-03 21:26:21 pesterhazy clojure -Sdeps '{:aliases {:a1 {:main-opts ["-e" "(println,:a1)"]} :a2 {:main-opts ["-e" "(println,:a2)"]}}}' -Srepro -M:a1:a2 will print out only :a2 #2018-12-03 21:26:42 dominicm Correct. #2018-12-03 21:27:15 dominicm Best bet is a custom main, and use jvm opts to compose. #2018-12-03 21:27:18 pesterhazy Also you can only ever specify one -m/--main option #2018-12-03 21:28:09 pesterhazy To me, those two facts combined diminish the usefulness of aliases, because you often need to produce a side effect to make an alias effective #2018-12-03 21:28:42 pesterhazy As an example, to start an nREPL server, you need to call nREPL's start-server function #2018-12-03 21:29:46 dominicm Yep! This is a limitation from Clojure itself, inherited from Java. I've given this some thought, and I don't know what multiple mains look like.
In parallel presumably? #2018-12-03 21:29:48 pesterhazy So one person in the team uses nREPL so they, so they run,e e.g. clojure -M:nrepl #2018-12-03 21:31:19 pesterhazy That to me is how composition of aliases would help - in different situations you can enable different sets of features #2018-12-03 21:32:27 pesterhazy Using :main-opts with ["-e" "(do (require ...) (start-server))]" is a little crude but works-ish - except for the fact that main-opts don't compose #2018-12-03 21:33:28 dominicm You can call functions, as long as they exit, but nrepl main doesn't, else it would be fairly useless as a main #2018-12-03 21:33:32 pesterhazy If not through aliases, I'm not sure how I would accomplish the same thing #2018-12-03 21:34:41 dominicm Edge has a workaround for this already, it looks at the system properties, and loads things based on flags there. #2018-12-03 21:35:04 pesterhazy do jvm opts compose? #2018-12-03 21:35:44 dominicm Yep! That's what I was trying to get across 😊 #2018-12-03 21:36:53 pesterhazy Gotcha. It seems a little arbitrary though that jvm-opts get concatenated, whereas main-opts are replaced #2018-12-03 21:39:21 dominicm Jvm opts naturally concatenate in the underlying system. I think they're unordered.
Main opts is talking about clojure's main-opts though.
You wouldn't be able to do what you want with nrepl and other mains even if concatenation happened. #2018-12-03 21:39:52 pesterhazy why?! #2018-12-03 21:41:35 pesterhazy is the problem with concatenating main-opts that the ordering of different aliases is not well defined? #2018-12-03 21:52:11 pesterhazy @U09LZR36F I guess your idea is to use an alias like {:jvm-opts ["-Dclojure.run.foo=foo.core"]} and then have a main that scans through system properties and calls those in some order? #2018-12-03 22:01:46 dominicm @U06F82LES somrthing like that, yeah #2018-12-03 22:02:50 dominicm @U06F82LES nrepl.main blocks forever. So rebel would not run afterwards. #2018-12-03 22:10:51 pesterhazy nrepl.server/start-server doesn't block #2018-12-03 22:11:54 dominicm You're thinking of clojure mains opts, I'm thinking of plain mains. My mistake. #2018-12-03 22:12:15 dominicm I don't think you can pass -e and -m anyway. #2018-12-04 00:08:03 alexmiller Something that we have discussed is a “start” option that would invoke an arbitrary function #2018-12-01 18:28:29 pesterhazy Corfieldization is a fun exercise but a clearer way to achieve this effect would be great #2018-12-01 18:57:58 dominicm The comma thing is a bug afaik #2018-12-01 20:50:33 seancorfield I'm sure that if someone can come up with patches to the clojure shell script that eliminate Corfield, lots of people will be very happy. Me included. #2018-12-01 21:12:19 dominicm Fixing it is difficult I would guess. You have to figure out how to generate properly escaped strings from edn. #2018-12-01 21:27:15 seancorfield Mainly you need to be able to round-trip arbitrary strings from bash to file to bash again. #2018-12-01 21:27:29 seancorfield Portably 🙂 #2018-12-01 21:41:54 dpsutton Will the corfield comma work in a powershell implementation of clj? #2018-12-01 21:44:17 alexmiller Sean’s description is pretty good, happy to have someone figure that out #2018-12-01 21:44:38 alexmiller I suspect powershell is prob easier #2018-12-01 21:49:29 dottedmag At least it's bash, not an arbitrary POSIX shell. #2018-12-01 21:54:16 alexmiller One issue is with nested quotes, another is needing to write the second half in bash (first half is in Clojure) #2018-12-01 21:55:09 dottedmag @alexmiller if you can give me an example of what needs to be done, I'll try to stretch my shell muscles. #2018-12-01 22:25:50 alexmiller Sorry, boarding a flight but there is a ticket in TDEPS #2018-12-01 22:27:45 seancorfield (there may be others but that's the first one I found) #2018-12-01 22:29:15 seancorfield @dottedmag ^ yup, that seems to be the whole issue. #2018-12-01 23:23:34 pesterhazy I’ll take a look as well #2018-12-02 11:04:01 pesterhazy I'm trying to tackle TDEPS-56, but am having trouble building the project. I've discovered two repos, brew-install and tools.deps.alpha. In the former I can run cd src/main/resources && bash -x install.sh ~/tmp
but that fails because *.jar is missing #2018-12-02 11:06:13 pesterhazy Presumably the jar is built from the tools.deps.alpha repo, but am I supposed to copy that over to brew-install/src/main/resources ? #2018-12-02 11:21:33 pesterhazy Next up, I'm running into this issue: bin/clojure: line 174: $install_dir/libexec/clojure-tools-${project.version}.jar: bad substitution #2018-12-02 11:22:11 pesterhazy ${project.version} is not valid Bash syntax so that must be related to the maven "filtering" feature #2018-12-02 11:43:04 pesterhazy looks like I was confused - master in the brew-install repo is an outdated branch #2018-12-02 14:08:15 pesterhazy ok figured it out - in a nutshell you need to go through the package.sh script and before that mvn install tools.deps.alpha (also point the pom to the snapshot version you're testing) #2018-12-02 14:31:48 alexmiller yes, exactly #2018-12-03 17:38:39 seancorfield @U06F82LES I share Alex's concern about both the eval aspect and the lack of compatibility with existing cached .main files -- I wonder if the script could verify the file contents, ensuring it was all \c\h\a\r\a\c\t\e\r\s and only calling eval in that case? I think that would retain compatibility with existing files and also address the security issue? /cc @U064X3EF3 #2018-12-03 17:43:10 alexmiller I think we should not use eval. I think probably it would be best to read the file as an array and then array splat it into the command line. I think that largely fixes the spacing problem, but not sure what still needs to be solved for quoting. #2018-12-03 17:44:27 pesterhazy happy to try a different tack #2018-12-03 17:45:16 pesterhazy to read the file into an array from dash, we'll need to use a separator #2018-12-03 17:45:27 pesterhazy were you thinking of separating arguments by newline? #2018-12-03 17:46:45 alexmiller do you need a separator? I thought space was the separator #2018-12-03 17:47:38 pesterhazy right, then we'll need to escape space #2018-12-03 17:48:31 pesterhazy so we'd write this -co {:aot-cache\ true}
#2018-12-03 17:49:35 alexmiller do you? the whole idea here is to write the string with spaces, then read it back into an array (breaking on spaces), then drop that into the args list #2018-12-03 17:50:05 alexmiller I think that’s actually fine and the issue is really what happens when you have Clojure quoted things inside the args #2018-12-03 17:50:46 pesterhazy here's how I'm thinking it could it could work:
- in the clojure code, we'd replace " " with "\\ " and "\\" with "\\\\"
- in the bash code, we'd read the file char by char, replace "\\ " with a space, replace "\\\\" with "\\" and else when encounter an unescaped space, start a new argument #2018-12-03 17:51:31 pesterhazy we'd basically implement our own quote/unquote scheme #2018-12-03 17:52:10 pesterhazy the main point is that space-as-arg-separator and space-as-literal-space need to be distinguished in the bash code #2018-12-03 17:56:29 alexmiller newline separator might also be a better choice - I think you may be able to automatically get some of what we want by changing the IFS too #2018-12-03 17:57:09 alexmiller ideally, I think the main file should have as little markup as possible #2018-12-03 17:57:55 pesterhazy sure, though the main-opts strings may contain newlines as well #2018-12-03 17:58:08 alexmiller the less magic that’s in the Clojure code the better. we’re just creating portability problems the more coordination is required between these two pieces of code #2018-12-03 17:58:26 pesterhazy agreed #2018-12-03 17:58:41 pesterhazy maybe keep a Windows version in mind as well #2018-12-03 17:58:55 alexmiller I think new lines in the args is much less common than multi-word / quoting #2018-12-03 17:59:19 alexmiller I am keeping Windows in mind :) (I suspect powershell has better options for this stuff though) #2018-12-03 18:00:13 alexmiller I need to do some other stuff, checking out #2018-12-03 18:00:21 pesterhazy even newlines aren't common, it shouldn't break if they do appear #2018-12-03 18:00:32 pesterhazy i'll give it another try #2018-12-03 19:17:18 pesterhazy If that looks like the right direction, I'll prepare a new patch #2018-12-03 14:37:16 jeroenvandijk I'm having an issue creating an uberjar using tools.deps with boot-clj via boot-tools-deps. The clj files of my local and git dependencies are not included if I don't AOT these files. Is this the right place for such an issue? #2018-12-03 16:19:30 seancorfield My advice would be to use one of the uberjar utilities designed for deps.edn instead of Boot. #2018-12-03 16:22:20 seancorfield boot-tools-deps was intended as a simple bridge for starting with a Boot project and moving the primary dependencies into deps.edn - there are several inherent limitations of the approach. #2018-12-03 16:23:04 seancorfield At work, we moved off Boot completely. #2018-12-03 16:23:30 seancorfield @jeroenvandijk ^ #2018-12-03 16:30:55 jeroenvandijk @seancorfield Ah good to know, thanks 😬. Out of curiosity what are you using for uberjars? #2018-12-03 16:31:39 dominicm @jeroenvandijk I'll plug pack.alpha which I authored. Sean maintains a depstar fork. #2018-12-03 16:33:34 jeroenvandijk @dominicm cool thanks. What's the best choice for creating uberjar's? Capsule, onejar? #2018-12-03 16:34:17 jeroenvandijk (btw I tried pack.alpha, but it seems to force me to choose a main namespace as jar entry. Something i'm not used to with lein and boot-clj) #2018-12-03 16:35:37 dominicm @jeroenvandijk capsule works in more contexts. I'm surprised you say that, because pack works on the opposite, clojure.main is the default unless you set it. #2018-12-03 16:36:06 jeroenvandijk ok i'll have to try again 🙂 Maybe i gave up too quickly #2018-12-03 16:36:35 jeroenvandijk Thanks, i'll come back with the results tomorrow #2018-12-03 16:37:08 dominicm @jeroenvandijk feel free to bother me if something doesn't work. #2018-12-03 16:38:06 jeroenvandijk awesome, thanks! #2018-12-03 16:58:09 jeroenvandijk thanks! #2018-12-03 16:53:50 seancorfield We create all our production artifacts with that (and run them with java -cp path/to/the.jar clojure.main -m entry.point for them to start at entry.point/-main). #2018-12-04 12:30:11 borkdude does tools.deps support git deps that have an edn file in a subdirectory? do you have to reflect this in the git url? #2018-12-04 12:30:37 borkdude e.g. my-project/lib2, my-project/lib2 #2018-12-04 12:30:42 dominicm @borkdude you can use :deps/root to specify a sub-directory #2018-12-04 12:30:47 borkdude perfect! #2018-12-04 17:32:01 shaun-mahood Specifically for use with REBL and openjdk, is there a way to tell REBL bring in the correct version of JavaFX as part of the extra-deps? I don't actually need it at the moment (I switched to the Java version to get it to work), but it seems like it would be useful to prevent problems (I ran into it on 2 of my machines that I use regularly for Clojure work). #2018-12-04 17:33:30 seancorfield Maybe ask in the #rebl channel? #2018-12-04 20:48:36 dottedmag It's a pity that existing .main-files are lossy: both arguments are separated by spaces and spaces within arguments are written as spaces. #2018-12-04 20:49:10 dominicm @dottedmag you mean via :main-opts? #2018-12-04 20:49:23 dottedmag Any idea how to invalidate those? I could use readarray Bash builtin to read \0-delimited file into a Bash array. #2018-12-04 20:49:24 dottedmag @dominicm yes. #2018-12-04 20:52:31 dottedmag Hmm, I can change $ck, but that invalidate all existing cached files, no matter if :main-opts had had spaces in them or not. #2018-12-04 20:53:12 dottedmag @alexmiller ^ would that be acceptable? #2018-12-04 20:56:05 dottedmag @dominicm That's the ticket I'm working on. No, Bash cannot keep \0 in variables or arguments. #2018-12-04 20:56:44 dottedmag There is even a hack in Bash to use '' as a "delimiter" to denote "use \0 as a delimiter". #2018-12-04 20:56:55 dominicm Didn't realize you were down in the trenches with this 🙂 #2018-12-04 20:57:28 dottedmag Also, execve(2) API accepts NUL-terminated strings, so no way \0 can be passed through. #2018-12-04 20:58:52 dottedmag Well, the patch itself is not so complicated. The only remaining issue is backward compatibility with previously written .main files. #2018-12-04 20:59:17 dominicm On a personal note, it seems to me that updating clj may/should invalidate all my caches #2018-12-04 21:07:50 alexmiller it is supposed to, as the install-level deps.edn is included in the cache calculation #2018-12-04 21:08:14 alexmiller but “invalidate” != “delete” #2018-12-04 21:08:32 alexmiller as the idea of writing a script that goes around rm’ing your dirs scares the crap out of me #2018-12-04 21:09:00 alexmiller using a .main2 seems perfectly acceptable too #2018-12-04 21:16:08 timgilbert ...or if it would be acceptable to fall back to a tool written to handle this kind of thing, like awk or something #2018-12-04 21:18:53 timgilbert ...or if the string could be written to .main separated by commas, and then read into the array on the bash side via readarray -d , #2018-12-04 21:43:54 alexmiller would prefer to a) not add any additional deps if at all possible and b) have the clojure program write something as dumb and obvious as possible to the .main file #2018-12-04 21:56:50 dottedmag @alexmiller Only locations of install-level deps.edn files are included in the key of cache, not their contents, right? #2018-12-04 21:57:24 pesterhazy (and submitted a first pass at a patch) #2018-12-04 21:58:57 dottedmag @alexmiller How about this one? Or I can do .main2, it's even simpler. #2018-12-04 21:59:00 pesterhazy if I'm not on the right track, happy if others take a swing at this, but I wouldn't want effort needlessly wasted if possible #2018-12-04 22:00:06 dottedmag @pesterhazy Seems much easier to do \0-based output and read back using readarray like this: - main_cache_opts=($(cat "$main_file"))
+ declare -a main_cache_opts
+ readarray -d '' -t main_cache_opts < "$main_file"
#2018-12-04 22:00:32 dottedmag The only remaining change is to ignore .main and and read/write .main2 instead. #2018-12-04 22:01:27 dottedmag @pesterhazy t.d.a change is also trivial: --- a/src/main/clojure/clojure/tools/deps/alpha/script/make_classpath.clj
+++ b/src/main/clojure/clojure/tools/deps/alpha/script/make_classpath.clj
@@ -76,7 +76,7 @@
(when (.exists jf)
(.delete jf))))
(if-let [main-opts (seq (get (deps/combine-aliases deps-map (concat aliases main-aliases)) :main-opts))]
- (io/write-file main-file (str/join " " main-opts))
+ (io/write-file main-file (str/join "\u0000" main-opts))
(let [mf (jio/file main-file)]
(when (.exists mf)
(.delete mf)))))) #2018-12-04 22:02:18 dottedmag @pesterhazy I have to go right now, so I can't finish it at the moment. #2018-12-04 22:03:06 alexmiller sorry, I don’t have time to work on this atm but will hopefully have a slice of deps time in a day or two #2018-12-04 22:03:34 dottedmag @pesterhazy I don't mind if you pick it up from here, if you think this approach is good enough. #2018-12-04 22:04:52 alexmiller if there are two (or N) options, that’s fine, just describe the tradeoffs in a table (and separate the file impl aspect from the “not breaking existing stuff” aspect - those I think are orthogonal) #2018-12-04 22:07:01 pesterhazy the obvious downside of readarray is that it's not available in bash3, which is what ships with macOS #2018-12-04 22:07:40 pesterhazy I think that rules it out #2018-12-04 22:08:00 dottedmag Oh, too bad. #2018-12-04 22:10:48 pesterhazy also not sure how reliable reading NUL character in bash is - that's the kind of thing where bash if flaky #2018-12-04 22:12:17 dottedmag That's why they have readarray in first place. #2018-12-04 22:12:25 pesterhazy many languages with C strings have trouble with NUL characters in strings, for the obvious reasons #2018-12-04 22:12:27 dottedmag And special case for -d '' there #2018-12-04 22:13:09 dottedmag read -r -d '' should work with an explicit loop. #2018-12-04 22:13:53 dottedmag As of now all NUL-characters are stripped from .main when read by Bash, so no backward compatibility problems here. #2018-12-04 22:14:19 dottedmag Unless someone relied on NULs being stripped 😄 #2018-12-04 22:14:20 pesterhazy well try it - I didn't have luck reading the "\\ " character sequence using read #2018-12-04 22:14:54 pesterhazy no it's not backward compatible - it would turn potentially multiple arguments into a single argument, i.e. change the semantics #2018-12-04 22:15:22 pesterhazy I don't think there's a way around using a different file extension #2018-12-04 22:16:38 dottedmag @pesterhazy $ echo '00000000: 310032003300' | xxd -r | while read -r -d '' i; do printf ">$i<\n"; done
>1<
>2<
>3<
#2018-12-04 22:18:04 dottedmag @pesterhazy By backward compatibility I meant backward compatibility for code that relied on \u0000 passed from deps.edn unchanged to the app. But that didn't work, so we don't need to care about it. #2018-12-04 22:27:53 pesterhazy @alexmiller I've updated the ticket with a summary #2018-12-05 03:50:56 baritonehands not sure what the process is for evaluating these #2018-12-05 03:52:02 seancorfield My simple guide for AOT: do not do it. #2018-12-05 03:52:25 seancorfield AOT is both unnecessary and a giant pain in the ... #2018-12-05 03:57:40 seancorfield (for all my snarkiness, the tip about needing to create classes and having it on your classpath is a good one -- it bit me when I tried to manually compile stuff) #2018-12-05 03:58:24 baritonehands as much as it is not preferred, it is still necessary in some cases #2018-12-05 03:58:48 seancorfield @gfredericks mentioned the classes thing in his talk at Conj -- I think it's a widely not-known thing 😞 #2018-12-05 03:59:59 seancorfield Yeah, AOT is a necessary evil in some very rare cases. Knowing how to successfully compile stuff when building JAR files, for example. #2018-12-05 04:00:11 seancorfield Don't worry, I'm just very grumpy about AOT 🙂 #2018-12-05 04:02:53 baritonehands Ah @gfredericks fellow Chicagoan #2018-12-05 04:17:25 seancorfield Have you watched his talk from Conj? Definitely worth it. #2018-12-05 04:18:19 baritonehands Not yet, still working my way through all of them #2018-12-05 04:18:34 baritonehands got distracted by #adventofcode lately #2018-12-06 09:28:33 kirill.salykin hi,
I am trying to build skinny jar with pack and got this error:
Exception in thread "main" java.lang.IllegalArgumentException: No matching clause: [nil :jar]
at mach.pack.alpha.skinny$run_steps.invokeStatic(skinny.clj:107)
at mach.pack.alpha.skinny$run_steps.invoke(skinny.clj:99)
at mach.pack.alpha.skinny$_main.invokeStatic(skinny.clj:175)
at mach.pack.alpha.skinny$_main.doInvoke(skinny.clj:158)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.RestFn.applyTo(RestFn.java:132)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
#2018-12-06 09:28:58 kirill.salykin but jar is created #2018-12-06 09:29:08 kirill.salykin please advice what I am doing wrong #2018-12-06 09:29:41 kirill.salykin pack version:
{pack/pack.alpha {:git/url ""
:sha "ddbbdbd4001c75647deddaad0f8bb7dc954fcb6f"}}
#2018-12-06 09:32:29 kirill.salykin UPD: I switched to latest version in master and everything seems to work #2018-12-06 09:36:35 kirill.salykin one thing tho - skinnyjar stucks if target directory contains prev version… #2018-12-06 09:41:13 dominicm @kirill.salykin master is a pretty big rewrite of pack. This bug isn't terribly surprising 😊
What do you mean by "contains previous version"? #2018-12-06 09:41:36 kirill.salykin if you run it twice in a row #2018-12-06 09:41:49 kirill.salykin #2018-12-06 09:41:50 kirill.salykin rm -rf ./target/*
clojure -A:pack mach.pack.alpha.skinny
#2018-12-06 09:42:09 kirill.salykin this fixes it #2018-12-06 09:42:10 dominicm @kirill.salykin what is the output in the case that it gets stuck? I expect a thread will error something? #2018-12-06 09:42:24 kirill.salykin no output, just takes a lot of time #2018-12-06 09:42:35 kirill.salykin more then 1 minute on machine #2018-12-06 09:42:45 kirill.salykin i was not patient to wait longer… #2018-12-06 09:42:50 dominicm Hmm. I'll try and debug that today. I'm not sure what could cause that. #2018-12-06 09:43:11 kirill.salykin if you clean the target directory - everything works perfectly #2018-12-06 09:43:47 dominicm Maybe there's something accidentally recursive. #2018-12-06 09:45:30 kirill.salykin shall I create the issue? #2018-12-06 10:22:13 dominicm Feel free, I'll try and get round to it today 🙂 #2018-12-06 10:23:09 kirill.salykin will do, thanks for your help! #2018-12-06 10:42:15 dominicm @kirill.salykin I'm unable to reproduce what you're seeing 😟 I might need some more context, maybe it only happens under certain deps.edn files or something? #2018-12-06 10:42:55 kirill.salykin I’ll try to create some minimal repro case #2018-12-06 10:43:01 kirill.salykin thanks for looking #2018-12-06 10:43:44 dominicm I'll mirror this on github 🙂 Not sure which channel you prefer for tracking this #2018-12-06 10:44:18 dominicm fwiw, I tested against pack itself like this: clojure -m mach.pack.alpha.skinny --lib-dir /tmp/lib --project-path skinny.jar #2018-12-06 10:47:07 kirill.salykin maybe it is something in our project #2018-12-06 10:47:33 dominicm What does your :pack alias look like? #2018-12-06 11:01:44 kirill.salykin hm
:pack {:extra-deps {pack/pack.alpha {:git/url ""
:sha "4a00565dce29225a1dd2a810c438b90b14f9e111"}}
:main-opts ["-m"]}}}
#2018-12-06 11:01:54 kirill.salykin I am bit confused with empty main-opts #2018-12-06 11:02:05 kirill.salykin lemme try w/o main-opts at all #2018-12-06 11:02:22 dominicm @kirill.salykin it really just saves you from typing -m 😄 #2018-12-06 11:02:39 kirill.salykin so it doesnt matter? #2018-12-06 11:03:22 dominicm not particularly 🙂 #2018-12-06 11:11:31 kirill.salykin works on empty project
I assume it is something within my project #2018-12-06 11:36:36 dominicm if you can track down what's within your project, I'd be interested #2018-12-06 12:01:58 kirill.salykin I’ll try, but later. It seems not urgent, also it is nice to clean target directory anyway (for outdated libs for instance)
so not with rm -rf target/* it just works #2018-12-06 12:02:33 kirill.salykin btw, pack dies if target directory not created #2018-12-06 14:16:12 kirill.salykin Question regarding juxt/pack, do I understand correctly that using --lib-type keep with skinny-jar should keep clojure file in jar?
the issue I see now that git dependency packed as empty jar (there is no gen-class) and clojure complaints that can find file on classpath #2018-12-06 14:17:59 dominicm @kirill.salykin --lib-type keep means that any dependencies that are jars, will stay as jars, and git dependencies which are directories, will stay as directories. #2018-12-06 14:19:06 kirill.salykin is it by design that git dependency packed into empty jar? or do I missing something in config? #2018-12-06 14:20:39 dominicm that sounds wrong. I haven't seen that in my testing #2018-12-06 14:21:29 kirill.salykin will try to create something that can reproduce the issue #2018-12-06 14:21:31 kirill.salykin thanks! #2018-12-06 14:25:12 dominicm @kirill.salykin my mistake, I've reproduced #2018-12-06 14:25:14 dominicm working on it #2018-12-06 14:25:48 kirill.salykin Thanks! #2018-12-06 14:27:16 mccraigmccraig i've got a lein-modules based project in a monorepo which has common deps in the top-level project.clj and per-module deps in each module subdirs own project.clj. it builds and tests all modules in a sensible (poset) order and correctly outputs versioned jars for each module - is there a way of achieving similar behaviour for deps.edn based project ? #2018-12-06 14:27:50 dominicm @mccraigmccraig what's poset order? I would just loop over each directory with bash & use pack 😄 #2018-12-06 14:28:18 mccraigmccraig it figures out the correct order itself, based on the dependency graph #2018-12-06 14:28:56 dominicm @kirill.salykin I've figured it out, dumb typo #2018-12-06 14:33:47 dominicm @kirill.salykin Try 24863d1ff5a54bb7cc783ff62272212b51c8b316 #2018-12-06 14:35:37 kirill.salykin wow, it blazingly fast, thanks!
trying #2018-12-06 14:36:05 dominicm @kirill.salykin I go on holiday soon, and I've wrapped up work 🙂 I have a few spare cycles. #2018-12-06 14:38:07 kirill.salykin yay! works! thanks #2018-12-06 14:39:13 dominicm Great 🙂 #2018-12-06 15:09:50 kirill.salykin btw, stuck issue seems fixed as well #2018-12-06 15:14:38 dominicm oh, hmm 🤔 #2018-12-06 15:16:40 dominicm ¯\(ツ)/¯ #2018-12-06 19:13:07 rpkarlsson I ran into some trouble using java9+ and Cider with deps.
Deps builds abbreviated paths ("src", as specified in :paths) instead of full paths ("/home/rpkarlsson/Projects/mc-kartan/src/") as other classpath builders seem to do.
Cider with java1.8 and lower determines the classpath in a different manner and is not effected by this.
Are abbreviated paths something that could potentially lead to other problems and because of that should be fixed in deps or should abbreviated paths be expected in tooling?
More info at:
https://github.com/clojure-emacs/cider/issues/2533 #2018-12-06 22:35:01 dottedmag I've got the spaces work in :main-opts #2018-12-06 23:15:46 timgilbert Looks reasonable to me, personally, though I'd prefer newlines to NUL characters. #2018-12-06 23:40:01 pesterhazy ¯\(ツ)/¯ #2018-12-06 23:41:33 pesterhazy I’m not playing the game of uploading competing patches #2018-12-06 23:45:10 alexmiller if there are competing approaches, please summarize the tradeoffs #2018-12-06 23:45:27 alexmiller I need that to make decisions and I will have a slice of time to do so soon #2018-12-07 01:05:58 dottedmag @timgilbert Newlines -> escaping comes back. Not good. #2018-12-07 17:17:23 donaldball What tools do folk like to use to build and publish jar artifacts to maven repos, particularly private s3 maven repos? #2018-12-07 17:51:04 seancorfield For the actual "push", I use mvn deploy:deploy-file ... #2018-12-07 17:52:45 seancorfield (the same, nasty invocation that @U09LZR36F has in his blog post, wrapped up in a shell script -- that also updates pom.xml with the SHA of the latest commit so that Clojars can link back to the commit that created the release) #2018-12-07 17:57:57 donaldball Thanks for the pointers! #2018-12-07 20:23:42 timgilbert FWIW, I'm working on a tool to post-process the results of clojure -Spom to add in the groupId/artifactId and pull the version from a git tag, since checking in partly-generated stuff into source control bugs me #2018-12-07 20:25:03 timgilbert (with the goal of automating everything up to the jar creation and then using pack for the rest) #2018-12-07 19:14:38 harrigan If I create a pom.xml with the following dependency:
<dependency>
<groupId>org.lmdbjava</groupId>
<artifactId>lmdbjava</artifactId>
<version>0.6.1</version>
</dependency>
and run mvn dependency:tree I see lmdbjava with 10 transitive dependencies. If I create a deps.edn with the same dependency:
{:deps {org.lmdbjava/lmdbjava {:mvn/version "0.6.1"}}}
and run clj -Stree I see 9 transitive dependencies. com.github.jnr:jffi:jar:native:1.2.16:runtime is missing. Could this be an issue with tools.deps? #2018-12-07 19:28:59 harrigan Ah that’s it exactly — thanks for the link. #2018-12-10 14:10:20 Brendan van der Es Hi all, question: Suppose I'm in a REPL in the middle of a datomic session with large cache in memory and I need to pull in a new library to continue processing. Is 'add-lib' the only way to pull in this dependency without losing the in-memory state of the jvm session? In other words, is there any way to have the CLI preserve this session state in memory in between reboots for new deps? Assuming the answer to this is a negative and suppose 'add-lib' fails, any other ways to preserve datomic memory cache between repl reboots? #2018-12-10 15:56:00 Ben Hammond I know its probably too late for this repl session ... #2018-12-10 14:19:39 alexmiller probably a better question for #datomic but afaik, the answer is no #2018-12-10 14:36:28 Brendan van der Es Yea, was afraid of that. I'm exploring ideas for a more datomic-centric repl experience but would need to make this kind of reloading seamless. Thanks for the quick response; will try #datomic. #2018-12-10 15:03:52 jeremys Hello, did someone try to install clojure with nix on os x? I am not using homebrew atm and I was wondering if the clojure package from nix included the necessary stuff to run tools deps and cli #2018-12-10 15:32:53 alexmiller don’t know - it’s not built or maintained by the core team #2018-12-10 15:42:21 dominicm @jeremys I don't know package it is, but one of them does include tools deps. #2018-12-10 15:43:40 jeremys @alexmiller Ok thx! #2018-12-10 15:47:03 jeremys I am thinking it fetches the necessary tools, I was about to take a look at the brew formulae to get a sense where things might differ but the tool part of the url on clojure’s website and the deps mention in the nix package leads me to think it’s there. Thx. #2018-12-10 15:47:17 alexmiller seems fine. that’s a couple versions back but what it does is intentionally pretty dumb. #2018-12-10 15:49:48 jeremys I’ll take a look! thank you. #2018-12-10 16:26:42 jeremys Just for information the nix package comes with the tools deps machinery. #2018-12-10 22:41:16 rschmukler Hey all 🙂 . Is it possible to use my deps.edn to point to a github repo that is using a leiningen project.clj? I naively tried pointing at it but got hit with an error regarding detecting the manifest. Makes sense, and I realize it might be out of the scope of what tools.deps is trying to achieve... but also feels quite useful #2018-12-10 22:45:01 seancorfield tools.deps, which is what does the dependency resolution under the hood, doesn't know how to use a project.clj file. Perhaps open an issue against that repo and see if they'll add a deps.edn file? #2018-12-10 22:46:27 alexmiller or if it has no transitive deps, you can force it to be a deps.edn project by putting :deps/manifest :deps in the coordinate #2018-12-10 22:47:10 seancorfield Presumably, you could do that anyway and just add that project's deps to your own deps.edn file anyway @alexmiller? #2018-12-10 22:47:31 alexmiller yes, that should work #2018-12-10 22:47:39 alexmiller sneaky! :) #2018-12-10 22:47:41 rschmukler Thanks for the quick replies! Sadly, it's of almost no value but for its transitive dependencies (which I needed to update). I ended up just excluding its (outdated) dependencies and then manually updating the dependencies in my deps.edn #2018-12-10 22:48:04 rschmukler Thanks again for the fast responses #2018-12-11 10:23:35 borkdude Is there an overview of how people can get a REPL in their editor for a deps.edn based project (without adding nREPL-specific deps to the deps.edn, so by setting up an alias in their profile)? #2018-12-11 10:55:28 orestis Ah, you’re looking for a proper guide, not just an alias? #2018-12-11 10:55:38 orestis Here’s my cider-nrepl alias, FWIW: :cider-nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.4.5"}
cider/cider-nrepl {:mvn/version "0.18.0"}}
:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]} #2018-12-11 10:56:27 orestis I start my REPL by doing clojure -A:cider-nrepl and then connect CIDER to it via cider-connect. I think that any nREPL-aware editor could do the same? #2018-12-11 11:13:35 borkdude yeah, maybe I could just add that to the README #2018-12-11 17:10:14 richiardiandrea I do the same the ClojureScript one is very similar #2018-12-11 17:12:37 borkdude cool, that’s the page I was looking for #2018-12-11 17:12:52 borkdude I guess it has to be CIDER agnostic since some people use VSCode or whatnot #2018-12-11 17:14:32 richiardiandrea Yep well cider is just a middleware. Also keep in mind that Calva for VSCode does use the cider Middleware #2018-12-11 17:15:06 borkdude Honestly I don’t know how that all works, I just use it #2018-12-11 17:15:24 richiardiandrea :+1: #2018-12-11 20:04:12 pesterhazy @borkdude The problem with using an alias that sets "-m" "nrepl.cmdline" is that it doesn't allow you to also specify -r or -m rebel-readline.main #2018-12-11 20:05:26 pesterhazy To me an nREPL-connected editor is only one part - the other part is a terminal/rebel-readline based prompt #2018-12-11 20:06:48 pesterhazy That could be solved if nrepl.cmdline supported a --next-main or --chain argument, like --chain rebl-readlin.main. What it would do is to spawn the server in the background and then call the next main function #2018-12-11 20:07:36 pesterhazy It could even do clean-up when that chained main exits #2018-12-11 20:08:17 pesterhazy @alexmiller has mentioned a start option, which could be in the same area #2018-12-11 20:08:22 orestis +1 for that. I never got to try it but I think you can combine -e to evaluate something arbitrary with -m ? #2018-12-11 20:08:51 pesterhazy yes you can -e something -e something-else -m some.main, though -m must be last #2018-12-11 20:09:25 pesterhazy but those don't work if the settings are specified in different aliases - the latter alias will override the earlier one #2018-12-11 20:41:29 pesterhazy yes, a library could be used for this purpose #2018-12-11 21:06:18 borkdude @pesterhazy cool, if you have anything better to link to, I would update the README with it #2018-12-11 21:17:17 dominicm In edge I have this working #2018-12-11 21:18:54 dominicm It depends on a process in user.clj which searches for jvm options like load_xyz, which are then translated to namespaces to load. This means you can start arbitrary background services. #2018-12-12 01:59:13 potetm You can put those on your path, add namespaced aliases to your ~/.clojure/deps.edn, and you can run nrepl from any dir containing a deps.edn. #2018-12-12 16:12:09 timgilbert Say, what directories do I need to cache in between invocations to take advantage of tools.dep's caching? I'm running builds on circleci and I'm caching ~/.m2, ~/.gitlibs, and .cpcache after running clojure -A:foo -Spath, but even after the cached directories are restored, when it invokes clojure -A:foo I see it downloading a bunch of jars from clojars. #2018-12-12 16:15:40 timgilbert Oh, also caching ~/.clojure which has its own .cpcache #2018-12-12 16:33:41 dominicm M2 & gitlibs should be enough #2018-12-12 16:36:23 alexmiller the ~/.clojure/.cpcache is only used if you are running clj without a deps.edn, so probably doesn’t apply #2018-12-12 16:36:55 alexmiller and you would need the local .cpcache if you want to avoid the second pre-jvm hit of building the classpath #2018-12-12 16:37:21 alexmiller but all of those file checks are based on timestamps, so when you say “restored” I wonder if that changes the timestamps #2018-12-12 16:37:33 alexmiller particularly in relation to the deps.edn file #2018-12-12 16:54:30 timgilbert Hmm, yeah, not certain how it works. Maybe I can "negative touch" the deps.edn file to decrease its last-modified time #2018-12-12 17:15:24 alexmiller easier to touch the cpcache before you try to use clj? #2018-12-12 17:15:56 alexmiller I guess we could have a flag that was the opposite of -Sforce #2018-12-12 17:16:14 alexmiller that would use the cached cp regardless of the timestamps #2018-12-12 17:17:02 alexmiller although then you’d have to make sure you did something else to force any deps changes to be picked up #2018-12-12 17:42:09 timgilbert Just confirmed on circleci that my deps.edn winds up being newer than the .cpcache and .m2 directories. I presume that circleci saves the mtimes on cache and restores them on restore. I'll try recursively touching those directories as an initial workaround #2018-12-12 18:50:41 hiredman I dunno, the touch thing will likely work, but I am suspicious of deps.edn always being newer and you always download jars #2018-12-12 18:51:13 hiredman even if those files are older, it shouldn't cause you to re-download jars into your m2 #2018-12-12 18:57:35 hiredman yeah, if you are re-downloading jars over and over in to m2 that seems like the up stream issue and fixing it will take care of down stream #2018-12-12 18:59:12 hiredman I dunno what facilities circleci provides, but you might stick a ls -R ~/.m2 at the start of your build before running clojure #2018-12-12 19:28:12 timgilbert Hmm, the touch doesn't seem to have any effect:
ls -lRt --time-style=full-iso deps.edn .cpcache ~/.m2 || true
-rwxr-xr-x 1 circleci circleci 1660 2018-12-12 19:25:36.984493358 +0000 deps.edn
.cpcache:
total 36
-rw-r--r-- 1 circleci circleci 7989 2018-12-12 19:25:38.420533085 +0000 2750907078.cp
-rw-r--r-- 1 circleci circleci 24551 2018-12-12 19:25:38.420533085 +0000 2750907078.libs
-rw-r--r-- 1 circleci circleci 64 2018-12-12 19:25:38.420533085 +0000 2750907078.main
/home/circleci/.m2:
total 4
drwxr-xr-x 15 circleci circleci 4096 2018-12-12 19:25:38.380531978 +0000 repository
/home/circleci/.m2/repository:
total 52
drwxr-xr-x 3 circleci circleci 4096 2018-12-12 19:25:38.388532199 +0000 aopalliance
drwxr-xr-x 3 circleci circleci 4096 2018-12-12 19:25:38.388532199 +0000 commons-io
#2018-12-12 19:29:13 timgilbert Same results with ls -lRtc to get the ctime #2018-12-12 19:42:15 timgilbert ...hmm, might be a PEBKAC problem #2018-12-12 19:50:35 timgilbert Argh, turns out it was a problem in my config where I changed the cache key things were being saved into but not the one where they were being restored from. Sorry for the noise! Once I fixed that up everything worked as expected and the deps are not being redownloaded in the second step. #2018-12-12 19:52:12 alexmiller yay #2018-12-13 00:05:22 Nolan hey everyone! have been loving tools.deps and have had no issues until i abruptly started receiving Error building classpath. with no apparent configuration change. has anyone seen this? i’ve been using deps in this way for a few weeks. anyone have any ideas on how to start debugging? #2018-12-13 00:15:17 Nolan for the next person: ssh-add -l may reveal that the identity previously being used is no longer there. still not sure exactly why that happened #2018-12-13 00:24:52 alexmiller maybe your ssh-agent died and restarted? #2018-12-13 01:38:07 Nolan must’ve. but it’s happily back at it now. i really appreciate what tools.deps has done for us. can’t say it enough—thank you for all your work on it. #2018-12-13 00:28:00 richiardiandrea when I create clojure -Spom can I tell clojure not to include org.clojure/clojure? I don't have it in my project deps.edn #2018-12-13 00:32:28 hiredman clojure merges in .clojure/deps.edn #2018-12-13 00:32:44 hiredman -Srepro Ignore the ~/.clojure/deps.edn config file #2018-12-13 00:33:18 richiardiandrea uhm I have in the back my head a voice that says that that does not work... #2018-12-13 00:33:22 richiardiandrea will try it #2018-12-13 00:34:23 hiredman it will likely break other things if you don't include a dependency on clojure somewhere #2018-12-13 00:34:42 richiardiandrea I am trying to build a pom.xml for ClojureScript so it should be fine #2018-12-13 00:35:05 richiardiandrea yeah it still adds
<dependency>
<groupId>org.clojure</groupId>
<artifactId>clojure</artifactId>
<version>1.9.0</version>
</dependency>
#2018-12-13 00:35:31 hiredman are you sure something else isn't puling it in? #2018-12-13 00:36:47 timgilbert I was just looking at this bit of code, I’m pretty sure -Spom will include your system-wide deps too, eg /usr/local/lib/clojure/deps.edn or whatever it’s called #2018-12-13 00:37:07 richiardiandrea well, I run clojure -Srepro -Spom after having deleted it...my deps.edn seems clean...dunno... #2018-12-13 00:38:49 timgilbert On the subject of pom generation, I’ve just published garamond, which rewrites the pom.xml that -Spom generates with a version ID from git tags: https://github.com/workframers/garamond #2018-12-13 00:39:23 richiardiandrea maaaan that's awesome I was fiddling with scripts 😄 #2018-12-13 00:40:12 timgilbert I still have a few features I’m working on, but it’s ready for use. It uses itself plus juxt/pack and deps-deploy to publish itself to clojars sans a project.clj #2018-12-13 00:40:31 richiardiandrea using those as well at the moment :+1: #2018-12-13 00:41:07 richiardiandrea thank you so much, this will definitely simplify my scripts! #2018-12-13 00:41:13 hiredman checkout clojure -Srepro -Sdescribe #2018-12-13 00:41:40 timgilbert Awesome #2018-12-13 00:42:03 richiardiandrea oh /home/arichiardi/.local/lib/clojure/deps.edn is something I haven't created #2018-12-13 00:42:16 richiardiandrea and indeed it contains the clojure version above #2018-12-13 00:42:23 hiredman yeah #2018-12-13 00:42:37 richiardiandrea tricky trick! thank you! 😄 #2018-12-13 00:42:44 hiredman I think the install scripts create that with the idea that it is the "system" level deps.edn #2018-12-13 00:42:49 richiardiandrea can I just get rid of :deps in there? #2018-12-13 00:42:57 hiredman I don't see why not #2018-12-13 00:43:20 richiardiandrea lemme do that then 😄 #2018-12-13 00:43:40 hiredman or just delete the file #2018-12-13 00:44:18 richiardiandrea yeah that's right it works! #2018-12-13 00:46:04 richiardiandrea thank you very much @hiredman #2018-12-13 01:20:36 alexmiller note that installing / upgrading clj will rewrite that file #2018-12-13 01:33:46 seancorfield Deleting the "system" level deps.edn will remove the default Maven repos and a few other things, right? #2018-12-13 01:34:42 seancorfield Yeah, :mvn/repos and :paths (which defaults to src)... #2018-12-13 07:46:40 borkdude is tools.deps officially supported on Windows as well? I didn’t think about this when choosing it as the thing for Advent of CLJC but I didn’t hear any complaints yet #2018-12-13 09:13:18 dominicm Not yet it isn't #2018-12-13 10:36:23 borkdude I hear a lot of people are using WSL on Windows now #2018-12-13 13:02:43 seancorfield Yup I do nearly all my clojure development on windows via wsl because of that. #2018-12-13 16:15:20 norman Is there a reliable non-brew way to get clj on OS X? #2018-12-13 16:26:07 alexmiller we are only providing brew as a means of install, but you could basically use the linux instructions too I think - they’re both basically doing the same thing #2018-12-13 16:26:22 alexmiller it’s essentially just putting files in the right places #2018-12-13 18:21:59 delaguardo Hi there. I have some problems with pulling git deps from private repo. My ssh-agent is configured and running but I'm getting USERAUTH fail every time. Is there some corner cases where it might fail that I must be aware of? #2018-12-13 18:28:21 seancorfield Note that it links to three related git/auth tickets. #2018-12-13 18:29:08 alexmiller it’s a star-shaped room, so mostly corners #2018-12-13 18:29:28 alexmiller so if you do ssh-add -l - do you see your identity? #2018-12-13 18:30:15 delaguardo Yes, and I can clone that repo. But for some reason jgit throwing an exception #2018-12-13 18:30:34 alexmiller other common issues are having stuff you don’t need in ~/.ssh/config #2018-12-13 18:30:41 alexmiller and issues with host key types #2018-12-13 18:30:50 alexmiller the former being more common in my experience #2018-12-13 18:31:19 alexmiller if you can dump your config here, I can eyeball it (can’t remember off-hand the offending things) #2018-12-13 18:31:33 alexmiller but I would know it if I saw it #2018-12-13 18:32:46 delaguardo Hm.. everything is working on another laptop. Only one difference - my key has passphrase and it is added to agent via osx keychain #2018-12-13 18:33:45 alexmiller that should be ok. can you dump your config file? #2018-12-13 18:34:26 alexmiller I think it’s IdentityFile that causes problem in your config (and you should delete that line) #2018-12-13 18:34:56 alexmiller so something like this should be sufficient: #2018-12-13 18:34:59 alexmiller Host *
AddKeysToAgent yes
UseKeychain yes
#2018-12-13 18:39:32 delaguardo Will try to do that. Thanks! #2018-12-13 18:40:23 alexmiller the macos version you’re using on the two machines can also make a difference - they have changed the keychain integration a couple times #2018-12-13 22:26:43 manderson @alexmiller RE: the AUTHFAIL issue, I added a patch to https://dev.clojure.org/jira/browse/TDEPS-49 that fixes the problem without having to remove the IdentityFile line in your config. Not sure if you saw it as it was just before Conj, but wanted to mention it. #2018-12-13 22:39:07 alexmiller Didn’t see it, thx #2018-12-14 06:22:30 didibus Someone brought up that tools.deps can't transitively resolve project.clj dependencies. Is that true? #2018-12-14 06:28:28 orestis AFAIK, yes. #2018-12-14 06:32:47 didibus Oh, so until all lib upgrade, it could cause some issues there #2018-12-14 06:32:56 didibus to add deps.edn #2018-12-14 06:33:21 didibus Can it handle pom.xml ? #2018-12-14 06:33:26 didibus Or only deps.edn ? #2018-12-14 07:00:07 orestis I think Pom is fine, but not 100% sure. I believe Pom is used for maven-things so presumably it works fine. #2018-12-14 13:47:23 alexmiller That’s correct #2018-12-14 07:00:41 orestis A usual trick is to fork a git library, at a deps.edn and use your fork with updated dependencies until it’s fixed upstream. #2018-12-14 07:36:41 dominicm Just to be clear, this only affects git dependencies. Nobody would depend on a git repo with only a project.clj. #2018-12-14 07:38:18 dominicm It is absolutely fine if you depend on a git repo, which depends on a leiningen-built project on clojars or maven central. #2018-12-17 23:01:34 martinklepsch When passing some serialized EDN to clj is that when people use commas instead of spaces to get it regarded as a single argument? #2018-12-17 23:01:52 martinklepsch clj -m some.ns '{:my "data"}'
#2018-12-17 23:14:27 dominicm Yeah. But not for the above case. #2018-12-17 23:14:35 dominicm It's inside of deps.edn files. #2018-12-17 23:42:04 aaelony Hi - longtime leiningen user here. I’m on a mac, I am using home brew, but the clj that gets installed stacktraces miserably #2018-12-17 23:42:32 aaelony Here is the stacktrace: $ clj -Sdescribe
Error building classpath. EOF while reading
java.lang.RuntimeException: EOF while reading
at clojure.lang.Util.runtimeException(Util.java:221)
at clojure.lang.EdnReader.readDelimitedList(EdnReader.java:746)
at clojure.lang.EdnReader$MapReader.invoke(EdnReader.java:680)
at clojure.lang.EdnReader.read(EdnReader.java:145)
at clojure.lang.EdnReader.read(EdnReader.java:111)
at clojure.edn$read.invokeStatic(edn.clj:35)
at clojure.edn$read.invoke(edn.clj:14)
at $slurp_edn.invokeStatic(io.clj:29)
at $slurp_edn.invoke(io.clj:22)
at clojure.tools.deps.alpha.reader$slurp_edn_map.invokeStatic(reader.clj:38)
at clojure.tools.deps.alpha.reader$slurp_edn_map.invoke(reader.clj:35)
at clojure.tools.deps.alpha.reader$slurp_deps.invokeStatic(reader.clj:58)
at clojure.tools.deps.alpha.reader$slurp_deps.invoke(reader.clj:54)
at clojure.core$map$fn__5587.invoke(core.clj:2745)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.Cons.next(Cons.java:39)
at clojure.lang.RT.boundedLength(RT.java:1785)
at clojure.lang.RestFn.applyTo(RestFn.java:130)
at clojure.core$apply.invokeStatic(core.clj:659)
at clojure.core$apply.invoke(core.clj:652)
at clojure.tools.deps.alpha.reader$merge_deps.invokeStatic(reader.clj:73)
at clojure.tools.deps.alpha.reader$merge_deps.invoke(reader.clj:70)
at clojure.tools.deps.alpha.reader$read_deps.invokeStatic(reader.clj:78)
at clojure.tools.deps.alpha.reader$read_deps.invoke(reader.clj:75)
at clojure.tools.deps.alpha.script.make_classpath$combine_deps_files.invokeStatic(make_classpath.clj:47)
at clojure.tools.deps.alpha.script.make_classpath$combine_deps_files.invoke(make_classpath.clj:43)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:68)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.core$apply.invokeStatic(core.clj:657)
at clojure.main$main_opt.invokeStatic(main.clj:317)
at clojure.main$main_opt.invoke(main.clj:313)
at clojure.main$main.invokeStatic(main.clj:424)
at clojure.main$main.doInvoke(main.clj:387)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:702)
at clojure.main.main(main.java:37)
#2018-12-17 23:42:45 aaelony any ideas welcome #2018-12-17 23:42:55 aaelony (or I’ll go back to leiningen #2018-12-17 23:42:56 aaelony ) #2018-12-17 23:43:21 seancorfield What does cat ~/.clojure/deps.edn show? #2018-12-17 23:43:55 dpsutton when i run clj -Sdescribe it gives me some output that includes config-files ["/usr/local/lib/clojure/deps.edn" "/home/dan/.clojure/deps.edn" ]. There are some deps.edn files and it seems like one of these is broken. can you check your equivalent ones #2018-12-17 23:43:57 aaelony That’s likely the culprit… $ cat ~/.clojure/deps.edn
;; The deps.edn file describes the information needed to build a classpath.
;;
;; When using the `clojure` or `clj` script, there are several deps.edn files
;; that are combined:
;; - install-level
;; - user level (this file)
;; - project level (current directory when invoked)
;;
;; For all attributes other than :paths, these config files are merged left to right.
;; Only the last :paths is kept and others are dropped.
{
{:aliases
{:new {:extra-deps {seancorfield/clj-new
{:mvn/version "0.5.5"}}
:main-opts ["-m" "clj-new.create"]}
}
;; Paths
;; Directories in the current project to include in the classpath
;; :paths ["src"]
;; External dependencies
;; :deps {
;; org.clojure/clojure {:mvn/version "1.9.0"}
;; }
;; Aliases
;; resolve-deps aliases (-R) affect dependency resolution, options:
;; :extra-deps - specifies extra deps to add to :deps
;; :override-deps - specifies a coordinate to use instead of that in :deps
;; :default-deps - specifies a coordinate to use for a lib if one isn't found
;; make-classpath aliases (-C) affect the classpath generation, options:
;; :extra-paths - vector of additional paths to add to the classpath
;; :classpath-overrides - map of lib to path that overrides the result of resolving deps
;; :aliases {
;; :deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.5.460"}}}
;; :test {:extra-paths ["test"]}
;; }
;; Provider attributes
;; :mvn/repos {
;; "central" {:url ""}
;; "clojars" {:url ""}
;; }
} #2018-12-17 23:44:14 hiredman what does clojure -Srepro -Sdescribe show? #2018-12-17 23:44:26 dpsutton it looks like aliases is malformed #2018-12-17 23:44:33 aaelony yeah #2018-12-17 23:44:44 dpsutton @hiredman s/he can't get any output due to this being unbalanced #2018-12-17 23:44:54 seancorfield That's where your EOF is coming from @aaelony -- you're missing a } #2018-12-17 23:45:34 hiredman @dpsutton right, clojure -Sdescribe would error if either the system or user level deps.edn had a problem, clojure -Srepro -Sdescribe would error if it was the system level #2018-12-17 23:45:37 aaelony Can I nuke the ~/.clojure/deps.edn and try again? #2018-12-17 23:45:51 seancorfield {:aliases
{:new {:extra-deps {seancorfield/clj-new
{:mvn/version "0.5.5"}} ; NEED EXTRA } HERE
:main-opts ["-m" "clj-new.create"]}
} #2018-12-17 23:45:53 dpsutton ah thanks for explaining @hiredman #2018-12-17 23:46:53 aaelony got it - that is now fixed #2018-12-17 23:46:54 aaelony thanks #2018-12-17 23:48:12 aaelony yay - all set now… phfew. #2018-12-17 23:48:17 aaelony big thanks all around #2018-12-17 23:50:45 seancorfield Happy to help! At work we drive everything with clojure CLI -- we started with Leiningen back in 2011, switched to Boot at the end of 2015, and switched completely to the new CLI stuff this year. #2018-12-17 23:51:33 aaelony Great - I’ve been mostly using R for the past year++ and getting back into Clojure and updating my toolset from lein… #2018-12-17 23:52:33 seancorfield Cool. And you can get started with Clojure 1.10 since the "gold" release of that dropped today which greatly improves error reporting in a lot of cases! #2018-12-17 23:53:22 aaelony yeah - ironically, I didn’t feel the error reporting pain (until just now). The clj tool is still on 1.9 until it gets released to home brew. #2018-12-17 23:56:52 alexmiller This particular error isn’t going to be any different with 1.10 afaik #2018-12-17 23:57:24 aaelony for me, this was a “not knowing what you don’t know” kind of error #2018-12-17 23:57:48 alexmiller The error reporting differences are primarily in the repl and this is just throwing an exception out of the main #2018-12-17 23:58:25 aaelony perhaps if there was a spec check on the ~/.clojure/deps.edn file #2018-12-17 23:58:34 alexmiller It would probably be reasonable to better trap around config file reading and give a better semantic error here #2018-12-17 23:59:00 alexmiller Well, we’ve been supporting back to Clojure 1.8 in tools.deps, which is pre spec #2018-12-17 23:59:10 alexmiller I will probably bump that up soon #2018-12-17 23:59:12 aaelony I see #2018-12-17 23:59:32 alexmiller But in this case, it didn’t even read so would never be able to do a spec check anyhow #2018-12-17 23:59:38 aaelony Though I read about it in the docs, I was pretty much unaware that the ~/.clojure/deps.edn file was involved #2018-12-18 00:00:14 aaelony I should have known better #2018-12-18 00:01:02 alexmiller Error could be more informative for sure #2018-12-18 00:02:14 aaelony 🙂 #2018-12-18 20:16:34 kenny Does :deps/root work for both local and git deps? #2018-12-18 20:21:30 kenny It appears it doesn't. #2018-12-18 21:12:51 dominicm Which doesn't it work for? It doesn't make sense for local dependencies really. #2018-12-18 21:51:19 alexmiller should work for both #2018-12-18 21:51:45 alexmiller in the case of a local dep it’s a directory relative to :local/root #2018-12-18 21:57:32 kenny It doesn't appear to work for local/root. I get the following message:
Error resolving /home/kenny/compute_software/app-ions: Manifest type not detected when finding deps for compute/alerts-api in coordinate {:local/root "../alerts", :deps/root "api", :sha "ca8f741dd264fa13a900cb1d283e32a21f4c21c8"}
#2018-12-18 21:58:13 kenny If I change :local/root to ../alerts/api it works as expected. #2018-12-18 22:03:36 alexmiller the message is saying that it’s looking for a manifest file (deps.edn or pom.xml) at the specified location and not finding one #2018-12-18 22:04:29 dominicm Which location is it looking in? Does it consider deps/root when searching? #2018-12-18 22:04:59 alexmiller it’s looking at <:local/root>/<:deps/root> #2018-12-18 22:05:28 alexmiller so it should be looking for ../alerts/api/deps.edn here #2018-12-18 22:05:31 dominicm That wouldn't explain it working with local/root alone? #2018-12-18 22:06:00 alexmiller well, the question is where .. is relative to #2018-12-18 22:06:19 alexmiller currently, it’s always relative to the current directory of wherever you are running #2018-12-18 22:06:41 alexmiller not the relative directory of whatever is depending on it #2018-12-18 22:06:58 alexmiller this is a known issue with local roots and relative dirs #2018-12-18 22:07:58 alexmiller I feel like I’m missing important parts of the problem description here so just kind of guessing #2018-12-18 22:09:16 kenny #2018-12-18 22:12:04 kenny Actually that's not correct.
pwd
/home/kenny/compute_software/app-ions
clj -A:dev -Spath
Error building classpath. Manifest type not detected when finding deps for compute/alerts-api in coordinate {:local/root "../alerts", :deps/root "api", :sha "ca8f741dd264fa13a900cb1d283e32a21f4c21c8"}
#2018-12-18 22:13:00 kenny > so it should be looking for ../alerts/api/deps.edn here
And that file exists:
ls ../alerts/api/deps.edn
../alerts/api/deps.edn
#2018-12-18 22:21:33 hiredman I am not sure, but I suspect that coordinate results in it looking for /home/kenny/compute_software/app-ions/api/../alerts/ #2018-12-18 22:25:05 hiredman no I am not right, just confused #2018-12-18 22:57:50 alexmiller you’re not using that function #2018-12-18 23:03:34 hiredman oh, yep, further confusion, that looks like it does that kind of resolution, but then doesn't appear to actually be called anywhere in tools.deps #2018-12-18 23:06:17 alexmiller but I will actually go back and say it’s not combining the two paths - it’s really just using :local/root #2018-12-18 23:06:26 alexmiller I was misremembering the code that’s in the git extension #2018-12-19 17:20:28 seancorfield Upgraded clojure to the 1.10 release via brew and then wondered why my outdated dependencies checker was no longer working... and then realized that TDEPS-82 is not yet fixed! #2018-12-19 18:13:43 alexmiller I am working through some tdeps stuff right now… #2018-12-19 18:13:48 alexmiller will try to hit that one #2018-12-19 18:14:06 alexmiller planning to get to a release by end of week #2018-12-19 18:25:23 seancorfield Thank you! #2018-12-20 22:56:36 geraldodev It would be nice if clj show invalid edn parsing errors #2018-12-20 23:00:10 dominicm I think there is a ticket for that, it's been mentioned. I think one would be welcome if you can't find one, if you can, a vote helps prioritize. #2018-12-20 23:06:29 alexmiller I was planning to look at that for the next release #2018-12-20 23:06:46 alexmiller but there are limits to what we can do other than say “the edn is invalid” #2018-12-20 23:07:28 alexmiller since the edn file is a map, it either will read the whole thing, or it will fail (if you miss a brace for example) #2018-12-20 23:07:39 dpsutton and bang for the buck i suppose. EOF encountered while ... is the current message, right? #2018-12-20 23:07:48 alexmiller yeah #2018-12-20 23:08:07 dpsutton it seemed straight forward but maybe that is stockholm syndrome #2018-12-20 23:08:36 dominicm @alexmiller is this the same behaviour as for Clojure? Could we look at something like the figwheel error tool? #2018-12-20 23:08:41 alexmiller I’m planning to catch this case and make the error message better (and skip the stack trace) but it’s probably not going to actually help you in a lot of cases #2018-12-20 23:08:55 dominicm Probably good questions for JIRA 😊 #2018-12-20 23:08:56 alexmiller sorry, don’t have context on figwheel #2018-12-20 23:09:39 dominicm I think figwheel can print out a snippet and give you an ASCII arrow pointing at the error. #2018-12-20 23:09:43 alexmiller the errors I see reported are almost always invalid edn (usually a missing brace), not a valid map that is invalid #2018-12-20 23:10:25 alexmiller I think figwheel is doing that with spec-like validation on invalid values, but that’s not the error I’m describing #2018-12-20 23:10:56 alexmiller we have specs and (if we bump the min clojure version) can check the read map against the specs to get that, but again, that’s not where I see most issues #2018-12-20 23:11:55 geraldodev I was bitten by Map literal must contain an even number of forms. Even on a modest file size it's very pratical to be pointed to the exact spot. I've downloaded diogo149/lein-edn-validator to help me find . #2018-12-20 23:13:10 geraldodev @dominicm tons of issues. ppl are using tools.dep. That's good! #2018-12-20 23:13:58 dominicm @alexmiller figwheel also does something with the data coming out of cljs compiler, and can information related to this.
I suppose in theory tdeps can be even smarter if the data was available from clojure.edn #2018-12-20 23:56:21 alexmiller Well, I’m not planning to do that #2018-12-21 08:06:41 dominicm @alexmiller out of curiosity, would you accept it as a contribution via a third party library or similar? Assuming startup time didn't hurt too much. #2018-12-22 01:42:24 dominicm Is there any reason :default-deps shouldn't be available as a top-level key? My reasoning being that I actually want any transitive dependencies that are newer than what I specify to be used instead. Maybe default-deps doesn't quite fill that niche. But if I depend on clojure 1.8, and cognitect-aws depends on clojure 1.10, I'm happy for 1.10 to be used. Maybe a "linter" could inform on this 🤔 #2018-12-22 21:16:41 borkdude how does one force download of dependencies in a deps.edn project? #2018-12-22 21:36:09 ghadi -Sforce #2018-12-22 22:11:37 borkdude I figured, but that also opens a repl …
$ clojure -Sforce
Clojure 1.10.0
user=> ^D
#2018-12-22 22:12:36 borkdude also, that forces recomputation of the classpath, I only want to force the initial downloading of stuff, else return fast without doing anything extra #2018-12-22 22:13:54 borkdude it seems the problem solved itself on circleci, just do stuff and it will take care of itself 😉 #2018-12-22 22:22:22 ghadi -Sforce -Spath @borkdude #2018-12-22 22:22:44 ghadi That will just printout the computed classpath #2018-12-22 22:25:07 borkdude thanks #2018-12-29 16:28:22 borkdude is there something to update a deps.edn file to the latest git sha’s ? #2019-12-31 15:51:08 jmv hi all, do deps.edn git refs support repos with multiple subprojects? i'm wondering if i can specify a path into the repo to find the deps.edn file along with the sha #2019-12-31 18:06:40 richiardiandrea @jmv we use :local/root for that, not sure about sha #2019-12-31 18:12:12 seancorfield @jmv305 :deps/root should let you navigate into a subproject within a git repo. #2019-12-31 18:50:24 jmv thank you, i will give that a try! #2019-01-02 21:33:58 alexmiller tools.deps.alpha 0.6.474 and clj 1.10.0.408 are now available.
Changes:
* TDEPS-79, TDEPS-109 Fix infinite loop in resolving recursive deps tree (will often improve resolution perf in other cases as well)
* TDEPS-50, TDEPS-109 Fix handling of Maven classifiers during resolution
To explicitly specify a Maven classifier in a dep, specify it in the lib name as follows: groupId/artifactId$classifier
Note: the :classifier attribute in Maven dep coordinates is no longer supported and will now produce an error.
* TDEPS-107 Canonicalize exclusion groupIds in pom gen
* TDEPS-82 Ensure -Sdescribe doesn't trigger resolution
* Error handling improvements when Maven artifact is not found
* Error handling improvements for invalid syntax in deps.edn
#2019-01-02 21:34:45 seancorfield TDEPS-82 ❤️ ❤️ ❤️ #2019-01-03 07:39:17 Chris Is it necessary to reinstall from http://clojure.org to upgrade to these releases? #2019-01-03 13:31:52 alexmiller Yes? How else would you get them? #2019-01-03 14:55:29 borkdude some packages have an upgrade command (but this would still get the updates from http://clojure.org) #2019-01-03 18:24:16 Chris Yes, I was wondering if there was an update command or it even automatically upgraded the tools deps lib. #2019-01-03 11:49:07 geraldodev I tried the last tools.deps and I have a fulcro depified project that has nubank/workspaces {:mvn/version "1.0.2" :exclusions [com.cognitect/transit-java]}
With the new version it does not loads transit-java. The problem is that it is excluded from workspace but its a dependency of transit-clj (of fulcro) and shadow-cljs . It looks like the exclusion is for good (affect other packages). clj -Stree does not show transit-java. When I revert to 1.9.0.397 the error is gone #2019-01-03 13:31:18 alexmiller Can you be more explicit with a repro to make sure I understand? #2019-01-03 14:11:17 alexmiller @geraldodev ^^ #2019-01-03 14:11:28 alexmiller in particular, would love to have your full deps.edn #2019-01-03 16:11:51 geraldodev My deps.edn, good_tree.txt was produced with linux-install-1.9.0.397.sh
bad_tree.txt was produced with linux-install-1.10.0.408.sh. Look that there are no transit-java with the latter version. It was supposed to be there because it is a dependency of fulcro and shadow-cljs #2019-01-03 16:13:29 geraldodev The only deps missing is a variation of sean corfield's ~/.clojure/deps.edn, but I think it does not play any role on this problem #2019-01-03 16:23:33 alexmiller what is the command you’re running to get the trees? #2019-01-03 16:23:42 alexmiller just clj -Stree ? or using any aliases? #2019-01-03 16:27:33 geraldodev clj -Stree on the project directory #2019-01-03 16:27:45 geraldodev no aliases. #2019-01-03 16:28:19 geraldodev The resolution appears to be different on the most recent version #2019-01-03 16:28:22 alexmiller thx, I will take a look #2019-01-03 16:28:39 geraldodev np, thnx for your work on tools.deps #2019-01-03 17:23:23 alexmiller @geraldodev that’s an interesting case. there are basically multiple dependency paths to transit-clj 0.8.313 (all same version). The path in which it is first found and selected (nubank/workspaces -> com.cognitect/transit-clj) happens to exclude its transitive dep, com.cognitect/transit-java. #2019-01-03 17:24:37 geraldodev I checked the version. It was strange to me because it is the same like you saw #2019-01-03 17:25:32 alexmiller in simplest form: #2019-01-03 17:25:35 alexmiller tree:
A (excl C) -> B -> C
D -> B -> C
selected:
A, D, B
#2019-01-03 17:26:19 alexmiller whereas if B happened to be encountered via D first, C would be included #2019-01-03 17:27:15 alexmiller I know why what you’re seeing happened to change, but I think this problem likely existed on the prior version too, but via some other arbitrary ordering #2019-01-03 17:28:06 alexmiller I will need to think about this a bit #2019-01-03 17:28:16 geraldodev But you are ble to get transit-java on prior version #2019-01-03 17:28:35 alexmiller you happened to get transit-java on prior version #2019-01-03 17:29:13 geraldodev via arbitrary ordering. I get it. It's a coincidence... #2019-01-03 17:29:15 alexmiller but I don’t actually think you’re guaranteed that - it’s probably a result of the order the deps are being encountered #2019-01-03 17:29:34 alexmiller I’m not certain of that, but that’s my suspicion #2019-01-03 17:30:21 alexmiller I think what this dependency tree is saying is not self-consistent #2019-01-03 17:31:29 alexmiller that is, if you treated the deps as logical propositions, there is no solution that satisfies both excluding C under A and including it under D in that simple version above #2019-01-03 17:32:01 alexmiller given that it’s the same B in both cases #2019-01-03 17:33:29 alexmiller for example, if you run this same example in Maven, you get a match of the “bad” tree (no transit-java included at all) #2019-01-03 17:34:33 timgilbert Wouldn't you want it to be like:
tree:
A1 (excl C1) -> B1 -> C1
D2 -> B2 -> C2
selected:
A1, D2, (resolve-duplicate #{B1 B2}), C2
? #2019-01-03 17:34:52 alexmiller in this particular case, the versions of B and C in both branches are identical #2019-01-03 17:35:02 alexmiller so I omitted the versions #2019-01-03 17:35:29 alexmiller if the two B’s are identical, then it shouldn’t matter which one you choose #2019-01-03 17:36:44 alexmiller and exclusions are never version-specific #2019-01-03 17:36:58 geraldodev There are no explicity dependency of transit-java on lein's fulcro template. It appears to include it. #2019-01-03 17:37:26 alexmiller tree:
A1 (excl C) -> B1 -> C1
D1 -> B1 -> C1
selected:
A1, D1, B1
#2019-01-03 17:38:14 alexmiller is a version-specific example (and the exclusion intentionally does not have a version, as exclusions are not version-specific) #2019-01-03 17:38:54 timgilbert I guess I was thinking the semantics of exclusion were more "omit C from the entire transitive tree of A", not "globally exclude C from the tree" #2019-01-03 17:39:05 alexmiller it is the former #2019-01-03 17:39:50 alexmiller but I’m saying that omit C1 can’t be consistent with also saying include D1 which depends on B1 which depends on C1 #2019-01-03 17:40:04 alexmiller you’re saying both omit C1 and include C1 - we can’t do both #2019-01-03 17:40:58 timgilbert I see the contradiction, yeah... #2019-01-03 17:43:13 timgilbert I guess I'd just intuitively expect that in order to get the behavior where C is excluded, you'd have to do
tree:
A1 (excl C) -> B1 -> C1
D1 (excl C) -> B1 -> C1
selected:
A1, D1, B1
#2019-01-03 17:43:35 alexmiller yes, that is what you have to do in either clj or Maven #2019-01-03 17:44:07 alexmiller there have been requests for global exclusions as a feature but that’s not currently a thing #2019-01-03 17:45:13 alexmiller if anything were to be done on this, it would be to make it non-arbitrary #2019-01-03 17:46:17 alexmiller bascially if you have B and B (excl C), do not treat those as interchangeable - prefer one over the other #2019-01-03 17:46:35 alexmiller this gets tricky when combined with version comparisons though #2019-01-03 17:46:52 alexmiller and I’m not sure which one you would want to prefer #2019-01-03 17:48:23 alexmiller seems like geraldodev’s expectation is that you would prefer including it #2019-01-03 17:50:24 geraldodev yes, like lein does #2019-01-03 17:50:47 alexmiller lein relies on Maven’s selection, which is also order-dependent #2019-01-03 17:51:38 alexmiller it wouldn’t surprise me if the ordering affected what you got in lein too #2019-01-03 17:51:43 geraldodev if workspace is not picking a version for you and another package is so be it. because it needs to be there. the version being the same is irrelevant to a greater proposal of having the lib #2019-01-03 17:52:24 alexmiller this is not about version picking though, it’s about inclusion vs exclusion #2019-01-03 17:52:44 geraldodev but the exclusion was caused because of the version being the same #2019-01-03 17:55:03 alexmiller there is no set of dependencies that can satisfy the constraints stated, so the most correct answer here is actually to fail #2019-01-03 17:55:19 alexmiller we’re just pouring semantics over the top of a logically inconsistent case #2019-01-03 17:55:43 alexmiller we do that all the time when selecting deps though wrt versions #2019-01-03 17:56:05 hiredman the question is kind of, are exclusions global, or are exclusions local. are exclusions applied while determining what the transitive dependencies of a given dependency are, or are they applied later after all the dependencies have been determined an a graph has been created #2019-01-03 17:56:18 alexmiller that’s not the question #2019-01-03 17:56:22 alexmiller exclusions are not global #2019-01-03 17:56:29 alexmiller they are always contextual #2019-01-03 17:58:12 geraldodev lein deps :tree on template project after lein ancient :upgrade. It uses workspaces on 1.0.2 #2019-01-03 17:59:00 alexmiller the selection algorithm is attempting to satisfy (often contradictory) constraints while also balancing performance concerns #2019-01-03 17:59:07 geraldodev transit-java is being picked by transitive dependency of shadow-cljs #2019-01-03 17:59:53 alexmiller clj is intentionally not using the same algorithm as either lein or Maven #2019-01-03 18:00:39 alexmiller those both prefer versions that are closer to the top or first encountered at a sibling level #2019-01-03 18:00:45 alexmiller clj prefers the newest #2019-01-03 18:00:45 hiredman maybe I misstated it, but it sounds like the difference between the current observed behavior, and the behavior @geraldodev is suggesting is between applying exclusions before or after it has been determined that the B both A and D depend on is the same, which maybe isn't possible, but I described it as global because it seems like in order to know A and D depend on the same B you need a global view of the dependency graph #2019-01-03 18:02:55 geraldodev @alexmiller it's the newest and forbbidenn by one dependency #2019-01-03 18:02:55 alexmiller I think this discussion is missing too much context of what’s being considered to be correct #2019-01-03 18:04:32 alexmiller like when choosing versions, you have to consider what versions you’ve seen, whether a version is under an exclusion of a selected version, etc #2019-01-03 18:05:11 geraldodev @alexmiller I'm in no place to argue about this. I just expected the lein behaviour. #2019-01-03 18:05:27 alexmiller and for performance reasons, we’re doing this in one pass, which limits information and when you know things #2019-01-03 18:06:27 alexmiller I’m still contending that the overall dependency tree here is logically inconsistent and there is no solution that meets all constraints #2019-01-03 18:07:25 alexmiller I would not be surprised if lein / Maven both have order-dependent answers to this and depending how you order things, you could see transit-java either being included or not here #2019-01-03 18:08:14 alexmiller if that’s not true, then their selection algorithm is effectively placing some preference, either intentionally or accidentally #2019-01-03 18:08:40 alexmiller clj’s selection algorithm is different, and may or may not make the same choice #2019-01-03 18:09:19 alexmiller I would prefer it not to be order-dependent though as that’s generally a source of confusion #2019-01-03 18:09:57 alexmiller Maven’s algorithm relies on order as means of selection, so it has to be #2019-01-03 18:11:00 hiredman some of it is maybe ergonomic, having the exclusion listed on a particular dependency leads to the expectation (as incorrect as it may be) that the exclusion is somehow limited in scope to that dependency and won't effect dependency info from other deps #2019-01-03 19:55:15 kenny In order to use Neanderthal on a Mac you need to specify these exclusions: https://github.com/uncomplicate/neanderthal/blob/7cf295119527843a6d1e9c17161b8a7b2f741309/examples/hello-world/project.clj#L6-L7. I cannot figure out a way to do this in a deps.edn. I tried this:
{:deps {uncomplicate/neanderthal {:mvn/version "0.19.0"
:exclusions [[org.jcuda/jcuda-natives :classifier "apple-x86_64"]
[org.jcuda/jcublas-natives :classifier "apple-x86_64"]]}}}
which results in
Error building classpath. Could not find artifact org.jcuda:jcuda-natives:jar:apple-x86_64:0.9.0 in central ()
I also tried
{:deps {uncomplicate/neanderthal {:mvn/version "0.19.0"
:exclusions [org.jcuda/jcuda-natives org.jcuda/jcublas-natives]}}}
which results in the same error. Is there a way to get this to work with a deps.edn? #2019-01-03 20:24:17 alexmiller exclusions in Maven can’t use classifiers afaik (certainly the deps.edn version does not support it) #2019-01-03 20:28:53 alexmiller I would expect something like this to work {:deps {uncomplicate/neanderthal {:mvn/version "0.19.0" :exclusions [org.jcuda/jcuda-natives org.jcuda/jcublas-natives]}}} #2019-01-03 20:31:19 kenny That results in
Error building classpath. Could not find artifact org.jcuda:jcuda-natives:jar:apple-x86_64:0.9.0 in central ()
#2019-01-03 20:31:38 kenny Having the exclusions that are in project.clj works as expected. #2019-01-03 20:31:44 kenny … in a project.clj. #2019-01-03 20:31:50 alexmiller yeah, the error there is accurate - there is no such jar #2019-01-03 20:32:25 alexmiller I guess this could be the exclusion code not being classifier aware #2019-01-03 20:32:57 alexmiller so it doesn’t know to try to exclude the os-dependent dependency #2019-01-03 20:33:35 alexmiller assuming that’s the issue, it shouldn’t be hard to fix #2019-01-03 20:45:13 alexmiller yeah, that was it. I have a fix for it #2019-01-03 21:09:16 kenny Great! Should I still file a ticket? #2019-01-03 21:38:42 alexmiller yes please #2019-01-03 21:38:48 alexmiller or I can do it if you want #2019-01-03 22:59:19 kenny Ok will do in a bit. Is there a workaround for now? #2019-01-03 23:49:02 alexmiller No, don’t have one, sorry. I can rev a new version tomorrow with the fix though #2019-01-04 01:10:04 kenny That'd be great. What would I need to do on my end to get that to work? #2019-01-04 14:39:00 alexmiller I filed a ticket at https://dev.clojure.org/jira/browse/TDEPS-112 and am working on a release. You’ll need to either “brew upgrade clojure” if on mac or re-install if on linux once it’s out (will take a few hours) #2019-01-04 16:32:36 kenny Sorry, meant to file the ticket - yesterday was quite busy. Thank you Alex! #2019-01-04 17:41:56 alexmiller yes #2019-01-04 17:42:07 alexmiller usually takes up to 24 hrs #2019-01-07 22:49:25 kenny @alexmiller I am still getting the same exception after running brew upgrade clojure. #2019-01-07 22:53:17 kenny Actually I needed to open a new terminal. It appears to be working now. #2019-01-08 03:19:51 alexmiller :+1: #2019-01-03 23:06:34 kenny I am getting this error when using clojure on a different computer than my usual one.
Error resolving /Users/Mark1/Provisdom/math:
I can run git clone successfully. Why would the Clojure CLI not be picking this up? #2019-01-03 23:49:39 alexmiller It uses jgit, so is a different pile of stuff #2019-01-03 23:50:04 alexmiller Can’t say I have enough to go on there #2019-01-03 23:50:18 alexmiller What os? Ssh? #2019-01-03 23:50:30 alexmiller Private or public repo? #2019-01-03 23:53:14 kenny Mac OS. SSH. Public. #2019-01-04 00:27:34 alexmiller run ssh-add -l to see if the ssh agent is running and has an identity in it (you should get probably 1 line back) #2019-01-04 00:28:06 alexmiller given that it’s public, you could use the https url too, which would probably just work #2019-01-04 00:29:03 alexmiller if you have an ssh identity though, the likely problem areas are your /.ssh/config, or something with /.ssh/known_hosts or the host key types #2019-01-04 01:10:49 kenny I'll just switch the https for now and deal with it in the future if I need to access private repos. #2019-01-04 10:56:11 rickmoynihan FYI peeps there is a new version of lein-tools-deps that uses 0.6.474.
The main improvement is the TDEPS-82 fix which significantly improves runtime performance as we no longer resolve deps twice (Thanks Alex!). Also we no longer hardcode how we locate the clojure command line tool.
https://github.com/RickMoynihan/lein-tools-deps #2019-01-04 12:52:06 jeroenvandijk Thank you! #2019-01-04 13:11:57 rcustodio Do you guys use depstar more or boot with this? #2019-01-04 13:46:22 jeroenvandijk I haven't used it on our major projects (leiningen, still too much hassle to switch). On new projects I'm working on boot and juxt.pack. expect for uberjar-ing boot works as expected #2019-01-04 13:57:26 rcustodio I see, good to know, thanks #2019-01-04 17:05:13 seancorfield We switched from Boot a few months ago and our entire dev/test/build pipeline is now based on clj/`deps.edn` and uses my fork of depstar (which has bug fixes and also the ability to create "thin" JAR files such as you'd push to Clojars). #2019-01-04 17:11:24 rcustodio Is that fork public? #2019-01-04 19:31:32 seancorfield Yes, and deployed to Clojars for easy usage. #2019-01-04 19:33:21 rcustodio Thanks!! #2019-01-04 13:59:30 rcustodio I’ve added javax.xml.bind/jaxb-api {:mvn/version "2.3.1"} but I’m having this Error building classpath. Could not find artifact javax.xml.bind:jaxb-api:jar:2.3.1 in central () #2019-01-04 13:59:33 rcustodio Can anyone help me? #2019-01-04 13:59:51 rcustodio :deps {br.com.vikingmakt/augustus {:mvn/version "0.1.12"}
br.com.vikingmakt/njord {:mvn/version "0.3.0"}
cheshire {:mvn/version "5.8.1"}
clj-http {:mvn/version "3.9.1"}
clj-time {:mvn/version "0.15.0"}
com.novemberain/langohr {:mvn/version "5.0.0"}
com.novemberain/monger {:mvn/version "3.5.0"}
javax.xml.bind/jaxb-api {:mvn/version "2.3.1"}
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.1"}
org.clojure/clojure {:mvn/version "1.10.0"}
org.clojure/tools.logging {:mvn/version "0.4.1"}}
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}
#2019-01-04 14:00:09 rcustodio No idea what is wrong with it #2019-01-04 14:04:35 alexmiller that leading quote on “2.3.1” looks weird - maybe a smart quote? “2.3.1" #2019-01-04 14:04:56 alexmiller I would retype that :) #2019-01-04 14:05:43 rcustodio woooow…. thanks @alexmiller for some reason emacs was not showing any diff on quotes to me #2019-01-04 14:06:12 alexmiller when I copy pasted, it came over as quote-space in my terminal #2019-01-04 14:06:19 alexmiller ¯\(ツ)/¯ #2019-01-04 14:06:25 rcustodio I see.. next time I will try that, thanks #2019-01-04 17:08:48 orestis I’m using @seancorfield ‘s depstar and it works just fine. I might want to do AOT eventually but for now it does the job. #2019-01-04 17:09:53 seancorfield AOT is evil. Stay away! 😇 #2019-01-04 17:11:01 orestis I’ve seen you saying that before, but I’m not sure what the downsides are for things that go to production? Upside is less than half of the startup time (we went from 8s to 23s when switching from cambada to depstar). #2019-01-04 19:28:46 seancorfield AOT of an entire app as the last step before building an uberjar for production deployment is fine if you care about startup time. #2019-01-04 20:16:41 orestis Thanks! #2019-01-04 17:11:57 orestis I’d love to know what I’m missing :) #2019-01-04 17:19:06 avi I’m finding AOT useful for CLI tools #2019-01-04 17:28:51 ghadi depstar and AOT are not incompatible #2019-01-04 18:39:10 trevor Can I have a java (maven) project in github as a dependency with tools-deps? #2019-01-04 18:45:05 dominicm No #2019-01-04 18:45:06 alexmiller no #2019-01-04 18:45:29 alexmiller tools.deps / clj don’t do “builds” so there is no opportunity to compile the java source #2019-01-04 18:45:59 alexmiller you will need to build and deploy the Java source as an artifact somewhere and depend on it that way #2019-01-05 14:36:40 mkvlr not sure if that's something worth considering but the new $classifier syntax doesn't play nice with http://jitpack.io. #2019-01-05 15:02:46 alexmiller Why? If there’s an issue, then please provide details or a ticket. I don’t get how it could affect jitpack #2019-01-05 16:44:23 mkvlr ok, will do that Monday morning #2019-01-05 14:46:58 mkvlr and something seems not to be quite right yet for our use case (TDEPS-50) as we're seeing java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider. Not sure yet if it's related to the uberjar or what because it works in development. Will dig deeper on monday. #2019-01-05 15:01:54 alexmiller That sounds like a native lib issue. There is a separate ticket to consider support around native libs - tools like lein do a bunch of extra stuff to make that work and I’m not sure yet if we need to take that on inside deps #2019-01-05 17:55:11 mkvlr @alexmiller oh sorry, please forget both things I've written above, it was entirely my fault: in our production build script I was explicitly requiring an old tools.deps version that didn't understand the $classifier syntax yet 🙈 Now that I've fixed that it's all working great 👌 Sorry again, thanks for your pointer and your great work. Enjoy the rest of your weekend! #2019-01-06 06:13:25 duncanmak where’s the source code to the clj tool? #2019-01-06 06:32:31 lilactown clj calls clojure with rlwrap #2019-01-06 09:03:11 pesterhazy wrapper scripts in the brew-install repo (not only for macOS but also Linux0 #2019-01-06 11:44:07 alexmiller Those are the correct places #2019-01-06 18:11:20 duncanmak morning @alexmiller #2019-01-06 18:11:39 duncanmak i’ve been clicking around, trying to understand how clj and tools.deps work #2019-01-06 18:12:18 duncanmak afaict, it’s all java code, so my question is, where / when does it change the classpath? does it start the jvm twice? once to calculate the right classpath, and once to actually run the instance? #2019-01-06 18:13:44 duncanmak and also, i’ve been looking for the code of the procurers #2019-01-06 18:30:16 alexmiller Yes, it runs the jvm twice - once to compute the classpath then again to launch your program #2019-01-06 18:30:33 alexmiller Caching is used to avoid the first step when possible #2019-01-06 18:31:28 alexmiller The procurer code is all under tools.deps.alpha.extensions #2019-01-06 23:39:02 duncanmak @alexmiller ahh, okay, so it’s not as magical as I thought, great! #2019-01-06 23:39:15 duncanmak @alexmiller where can i see it running twice, is that in the shell script wrapper? #2019-01-07 00:07:19 seancorfield @duncanmak Line 179 is only run if you ask clj to resolve tags for you. Optional. But required before building the class path from git repos. #2019-01-07 00:10:07 duncanmak Cool #2019-01-07 00:10:15 duncanmak Very clever! #2019-01-07 00:10:36 duncanmak I'm working with Kawa Scheme, I'm hoping to build a similar thing for it also #2019-01-07 15:10:25 borkdude it it me or is tools.deps missing from this question in the Clojure Survey?
22. Which tools do you use to compile/package/deploy/release your ClojureScript projects? #2019-01-07 15:13:17 alexmiller that’s ClojureScript-specific - is that a thing? #2019-01-07 15:14:37 alexmiller I just added it, so it’s there if people want to pick it #2019-01-07 15:15:03 borkdude well, I do a lot of things with cljs.main, which uses tools.deps? #2019-01-07 15:15:18 borkdude like compiling. I deploy via scp. #2019-01-07 15:15:56 dpsutton be nice if cljs-test-display was added to the cljs testing question as well #2019-01-07 15:19:14 borkdude so maybe cljs.main should be the option instead of tools.deps, dunno 🙂 #2019-01-07 15:26:29 alexmiller I changed it to “clj / cljs.main” #2019-01-07 15:26:38 alexmiller what’s cljs-test-display? #2019-01-07 15:27:50 alexmiller that seems like a visualization tool rather than the test runner? #2019-01-07 15:30:40 dpsutton i suppose. then credit would go to figwheel and cljs.test for running the actual tests. but i feel like cljs-test-display is more than that. i just mention because that's my interface to all of our cljs tests and i was looking for it in the list #2019-01-07 15:30:51 dpsutton and this is the worst room for this sorry 🙂 #2019-01-07 15:34:12 alexmiller ha, no worries - there is an Other category for stuff like this - often new stuff shows up there and then I add it the following year if it’s strong enough #2019-01-07 20:14:43 duncanmak hey @alexmiller, i also started looking into Groovy’s Grape, which I see to be something very similar to tools.deps for Clojure #2019-01-07 20:15:19 duncanmak is the choice of using Ivy vs Maven (or Eclipse Aether) significant? Or do the two APIs basically do the same thing and it’s just a personal choice to use one or the other? #2019-01-07 21:00:31 alexmiller I’ve spent some quality time with Grape in the past and worked on a build system that used Ivy extensively as well, so I’m somewhat familiar with both. I have never looked at Ivy’s actual code to see if it could satisfy what we needed wrt Maven repositories (although I’d be somewhat surprised if it didn’t ultimately use the same Maven APIs we are already using). #2019-01-07 21:19:17 alexmiller I glanced at it and it’s actually not using the Maven APIs at all afaict, so would be an alternate path to investigate for downloads at some future time (could also jettison both and just do the moral equivalent directly in Clojure, which might actually be better). We do use a bunch of Maven stuff to read pom files and we’d need that regardless though. #2019-01-07 21:37:07 kenny When starting a REPL with clj, is there a way to set default JVM opts? #2019-01-07 21:39:38 alexmiller no, but that is a ticket in the system for consideration #2019-01-07 21:40:23 alexmiller you can pass them explicitly on the command line by prefixing with -J though like -J-Xmx1g #2019-01-07 21:41:06 kenny Hmm ok. Any idea what the default -Xmx is? #2019-01-07 21:42:39 alexmiller in latest JVMs there is no default - it’s computed as a fraction of system resources I think, or something like that #2019-01-07 21:43:00 alexmiller and more modern GCs can also release heap back to the OS #2019-01-07 21:43:39 alexmiller so no, we’re not setting any jvm settings - that’s up to you to determine #2019-01-07 21:46:14 kenny Got it, thanks. #2019-01-07 21:51:39 duncanmak @alexmiller thanks! #2019-01-08 14:49:45 john I seem to be stuck on betas with brew #2019-01-08 14:50:18 john I just ran brew upgrade clojure and it just updated with clojure 1.10.0.403 -> 1.10.0.411 #2019-01-08 14:50:39 john and
user=> *clojure-version*
{:major 1, :minor 10, :incremental 0, :qualifier "beta6"}
#2019-01-08 14:52:02 john my global deps.edn, duh :face_palm: #2019-01-08 15:21:38 borkdude yeah, that’s what -Srepro is for 🙂 #2019-01-08 15:57:36 alexmiller yeah, if you have a ~/.clojure/deps.edn, it won’t be modified (in case you’ve added your own config to it) #2019-01-09 11:41:49 gphilipp Let’s say I have a repo named foo with 3 logical modules in their own subdirectory: a, b and c. Each have their own deps.edn with their required dependencies. I can also have a root deps.edn which refers to these modules using 3 {:local/root ...} deps. Now, I fail to see how I can add a dependency using the git dep scheme on say, module foo-b from brand new project bar which lives in a separate repo. Has anyone done something similar before ? #2019-01-10 10:55:24 gphilipp Thx @alexmiller. While I’m at it, we’ve been bitten by https://dev.clojure.org/jira/browse/TDEPS-74 (deps transitivity issue involving :local/root). Currently the scenario whereby project foo depends on module a of project bar using bar {:git/url "" :deps/root "a"} and bar.a depends on siblings module bar.b using {b {:local/root "b"}} doesn’t work. @matthias.margush has put together a patch which solves the issue (more info on the ticket), if you have some time to look at it that would be great. We’re not far from being able to switch over to tools.deps for several multi-modules projects. #2019-01-10 11:41:52 alexmiller yeah, there are several issues with the path stuff and I don’t think the change there is sufficient to handle the real scope of the issues #2019-01-10 15:49:03 matthias That makes sense - it did seem like the way coordinate specifiers interact could use some hammock time overall. #2019-01-10 15:49:43 matthias What are the other jiras that relate? #2019-01-10 15:53:04 alexmiller some of those intersect with other problem areas (config file management) #2019-01-14 18:59:44 matthias Hi Alex. Reading through these in more detail, it looks fixes could potentially be bundled together with a fix for TDEPS-74... they are closely related, right? If it would be helpful at all, I'd be willing to take a look at these in context of the patch I started for TDEPS-74. #2019-01-10 15:53:30 alexmiller I’d say it’s unlikely I’m going to look at this set of things this week #2019-01-10 16:28:18 matthias Oh yeah, not expected. Just want to get it on the radar. tools.deps really opens up the ways that we can organize work across teams, and thanks for the work you all are doing on it. #2019-01-10 15:55:50 matthias One thing I noticed personally was that transport, location and manifest type are different things that relate and interact, but don’t map conceptually one to one to how extensions are dispatched. #2019-01-10 16:00:54 alexmiller transport and location aren’t uniform dimensions for procurers #2019-01-10 16:01:12 alexmiller manifest types are something that extensions dispatch on #2019-01-10 16:23:59 matthias Are procurers things like maven, local, and git (and not pom, deps)? #2019-01-10 16:55:27 alexmiller Yeah - there are two layers of multimethods #2019-01-10 17:36:02 matthias I see. #2019-01-10 17:43:38 matthias I think possibly related to your comment about uniformity of dimensions - a capability I’d like to be able to support for our engineers is to checkout a git/deps dependency locally while working on a cross cutting feature. External tooling could support this well enough for example by transforming a git coordinate to a local one in the local work area.
But the differences in the set (& maybe the semantics) of dimensions make the creation of tooling more bespoke than needed. I think specifically in this use case it would help for the :deps/root to work the same between git coordinates and local coordinates. (Unless that’s already the case and I misread the code.)
So such a tool needs to know the dimensions available and how they are used for each kind of coord. #2019-01-10 23:40:46 duncanmak @alexmiller i’m interested in learning more about how to download jars without using maven/ivy - do you know of anything that talks about how that works? #2019-01-10 23:52:14 seancorfield @duncanmak If you know the URL of the JAR in the repository, you can just download it via any means -- such as curl at the command line, or programmatically using URI and byte input streams etc. #2019-01-10 23:53:13 seancorfield (unless the repo is secured and then you'll need to factor in auth credentials) #2019-01-11 00:01:13 hiredman maven has different plugins for talking to repos, but the most common transport for talking to repos is http, so a maven repo can just be directory containing files on a webserver #2019-01-11 00:02:25 hiredman the directory tree structure in that case is actually the same structure as ~/.m2, so you can upload your m2 to s3, make it publicly accessible, then use it as a maven repo #2019-01-11 00:03:25 hiredman the complicated part of maven is parsing and understanding pom files #2019-01-11 00:12:35 duncanmak i see #2019-01-11 12:18:58 eval2020 Is dependency resolution in any way influenced by the contents of /.m2? Ie I’m considering caching /.m2 on CI but was wondering whether CI-runs without a local repository might actually yield helpful early warnings/errors (eg library updates that cause problems). #2019-01-11 12:24:42 jeroenvandijk Not caching would tell you if a dependency suddenly disappeared (which shouldn't happen for at least clojars), but otherwise I cannot think of a reason to do it. If the library is present locally it will not do an additional fetch unless you use -SNAPSHOT versions, which is probably not desirable anyway. Having said that i'm curious for other answers to this question #2019-01-11 12:34:39 eval2020 that’s indeed what I would expect. I see there’s also .cpcache/ which seems to contain the actual result of resolution. #2019-01-11 13:49:27 jeroenvandijk Actually now I come to think about git deps, where force pushes and deletion of repo's are a possibility. This might be a valid concern #2019-01-11 13:50:07 jeroenvandijk But git deps don't really interact with ~/.m2 deps i think #2019-01-11 15:29:21 eval2020 indeed - these end up in ~/.gitlibs and it seems you can’t set this path (as opposed to mvn/local-repo) so not easily cacheable on gitlab-ci. #2019-01-11 16:05:20 j0ni depending on your CI solution, you could add a full dependency pull to a step which depends on tests passing, so you don't waste time on it as part of your regular feedback cycle #2019-01-11 13:38:39 mike_ananev @alexmiller Hi. Is there any updates with cli tools for Windows? very expected feature #2019-01-11 13:39:49 alexmiller no update, sorry #2019-01-11 15:45:02 borkdude @mike1452 how is the WSL support in Windows nowadays? #2019-01-11 16:56:03 seancorfield clj/`deps.edn` works really well on WSL. The problem you then get is editor integration of you use a windows editor with your repl running on Linux. #2019-01-11 16:57:12 seancorfield (You can't "load file" from windows because Linux won't understand the path the editor sends to it) #2019-01-11 16:59:51 seancorfield There are workarounds but it's a less than ideal process. I tend to select-all + evaluate (and I have symlinks set up in WSL so the file paths match windows, modulo the drive colon prefix). #2019-01-11 20:51:07 timgilbert Say, anyone executing their applications via clojure inside of docker images? Want to share any findings? I'm wondering how to handle caching (ie, whether to clojure -Sforce -Spath to download deps to the image, then cache them for faster startup) #2019-01-11 21:23:22 mike_ananev @borkdude corporate windows policies prohibits using WSL. so, half of our team, with BYOD devices, using clj cli tools, and another half with corporate PCs (Windows OS only) should use leiningen, which is available for Windows and allowed in our network . it is very inconvenient. we are still waiting for the windows version of Clj cli tools. Well, "First rule of Open Source, never ask when something will be done". And it is sad. 😞 #2019-01-11 21:44:22 duncanmak hmm, interesting #2019-01-11 21:44:36 duncanmak what is making clj Unix-dependent? #2019-01-11 21:46:51 hiredman it needs a powershell port I guess, which is an issue lein had a long for time, basically the people writing the tools don't use windows much and don't use powershell, so it is difficult for them to do the port or review it #2019-01-11 21:53:56 dottedmag I had to port a bunch of shell code to PS recently, and it was not so difficult, but annoying as hell. Favourite tidbits: 1) downloading a file with a progress indicator enabled is awfully slow (it looks like this indicator gets updated for every byte, and terminal on Windows is quite slow), and this progress indicator is enabled by default; 2) extracting a file from Zip using PS standard library cmdlet is ~10 times slower than directly invoking .NET code this cmdlet is wrapping. #2019-01-11 21:55:02 dottedmag OTOH one does not have to worry about differences between supported grep options or check if the tool is called shasum or sha256sum. #2019-01-11 22:15:46 alexmiller I have a not quite working port of clj to powershell. In the journal I just posted I asked people interested in actually working on dev to ping me here. If someone(s) are interested in really digging in, let me know and I will do what I can to guide. #2019-01-13 03:24:39 ajoberstar @alexmiller I've got some PowerShell experience and would be interested in helping this along. What still needs to be done? #2019-01-12 04:29:11 alexmiller one tricky thing in the clj script is that the script gets modified on install to point to the absolute location of the lib dir #2019-01-12 04:29:26 alexmiller not sure what the best way to handle that is #2019-01-12 04:31:02 alexmiller would be happy to add it but I’m not going to have time to work on it, some whatever help you could provide would be useful #2019-01-12 04:33:11 alexmiller the current install is basically a tar file plus an install script that mostly just moves files around, but also makes the jar location replacement mentioned above #2019-01-12 06:29:53 alexmiller We don’t do PRs in this repo so patch on the jira would be better #2019-01-12 06:30:11 alexmiller And ca is required #2019-01-12 07:30:27 kenji_signifier thx @alexmiller and @seancorfield I’ll follow the workflow once I create a patch for it. #2019-01-13 08:59:50 dottedmag Is there a way to have global :jvm-opts? #2019-01-13 09:00:15 dottedmag Besides making a shell wrapper around clj. #2019-01-13 13:54:42 alexmiller Currently no, that’s an enhancement request #2019-01-14 19:32:31 borkdude since there is no cognitect channel and this is a tools.deps project, I’ll post it here. there’s a trivial PR lingering for a while now. I wonder why there’s no interesting in accepting it from Cognitect Labs:
https://github.com/cognitect-labs/test-runner #2019-01-14 19:33:24 borkdude I even forgot about the previous PR and made a duplicate one 🙂 #2019-01-14 19:36:08 alexmiller Sorry, Luke was minding that project and he’s not at Cognitect anymore #2019-01-14 19:36:33 alexmiller I’ll take a look when I get a chance #2019-01-14 19:37:30 borkdude cool 🙂 #2019-01-14 19:37:58 borkdude I’ve been using it more and more. There’s also a CLJS clone of it #2019-01-14 19:40:27 borkdude ghadi made the first PR. he is at cognitect right? #2019-01-14 19:57:13 dominicm @U050ECB92 is this any you became a space cat wizard? #2019-01-14 19:48:29 alexmiller He is #2019-01-14 19:51:09 ghadi merged @borkdude #2019-01-14 19:51:43 ghadi 209b64504cb3bd3b99ecfec7937b358a879f55c1 is master now #2019-01-14 21:03:36 avi Are there any examples out there of libraries that have both a project.clj and also a deps.edn, for compatibility with both Leiningen and tools.deps, but don’t duplicate the dependency specifications? I’m interested in possibly contributing such a feature to a library I use. #2019-01-14 21:05:34 dpsutton :plugins [[lein-tools-deps "0.4.1"]] is in one of our project.clj files. This allows the lein project to read deps out of the deps.edn file #2019-01-15 16:06:38 rickmoynihan :plugins [[lein-tools-deps "0.4.3"]] is the latest and should be a bit quicker at resolution. #2019-01-15 20:27:36 avi Thanks! #2019-01-15 21:04:09 thosmos I've been using that for quite a while, and it works great. but now Cursive supports deps.edn directly so I mostly use that #2019-01-16 09:52:36 rickmoynihan lein-tools-deps is not without its issues… I should probably make them clearer on the repo. #2019-01-16 13:52:19 avi Once I read the docs, I was up and running with no clear issues, at least not at first. Maybe some of the steps could be moved to the top of the readme, but no big deal #2019-01-16 13:56:46 rickmoynihan Good to know… but it can cause problems if you’re not aware of how it works… and it’s not made clear yet.
Essentially it currently flattens the whole transitive dependency graph. It does this to guarantee resolution is the same in both tools.deps and leiningen — but it does mean that downstream mvn resolution semantics can change. #2019-01-16 13:57:21 rickmoynihan e.g wrt to :exclusions #2019-01-16 14:45:14 avi That sentence seems very important… if it’s not in the readme already maybe just add it as-is? #2019-01-16 14:50:57 rickmoynihan I think it warrants a small section covering the implications tbh — will write it at some point. #2019-01-16 14:51:10 avi 💪 #2019-01-16 14:51:20 avi I’d be happy to review if that’d be helpful #2019-01-16 14:53:45 rickmoynihan you’re welcome to comment on what is written — but it’ll probably be ok. Too busy to do it just now though. #2019-01-16 14:53:52 rickmoynihan there are github issues that touch on it already #2019-01-16 14:54:05 avi got it! #2019-01-14 21:05:50 dpsutton Makes it very simple to use lein for building/deploying but deps.edn for dev and running #2019-01-14 21:39:37 avi Will check it out, thank you! #2019-01-15 00:14:55 lilactown how do you like using http://sr.ht? #2019-01-15 00:21:18 hiredman so far all I have done is create that repo and push to it, seems fine, I appreciate that I can browse the site without letting it execute js #2019-01-15 20:27:57 avi Great, thanks for sharing! #2019-01-15 00:13:31 richiardiandrea we are doing like the above, I have also a deps.edn only project but having lein makes it easy for colleagues to use Cursive, which still has some quirks with deps.edn #2019-01-15 00:43:12 cfleming @richiardiandrea It does? I’d like to hear about them. #2019-01-15 00:44:42 richiardiandrea ah ah for sure 😄 one of my colleagues opened a couple of issues so you are aware of everything I think, don't remember which ones....wherever you see monorepo is us I guess! #2019-01-15 21:46:52 cfleming Ok, I’ll check those out, thanks - AFAIK most things should be fixed these days. #2019-01-15 15:16:04 vemv 👋
I asked this yesterday in off-topic, wondering what you think about it https://clojurians.slack.com/archives/C03RZGPG3/p1547483045535400
One thing I would fear if people widely adopted "SHAs for every dependency" is that two different libraries (let's call them A and B) could independently declare a dependency on C, but at different SHAs.
Is that a scenario that tools.deps can gracefully handle? #2019-01-15 15:16:21 alexmiller yes #2019-01-15 15:16:38 alexmiller well depends what part you’re asking about #2019-01-15 15:16:56 alexmiller if you declare two sha’s in the transitive dep tree, it will use the newer one #2019-01-15 15:21:30 vemv thank you! I'll look into using tools.deps at work then 🙂 #2019-01-15 15:25:08 alexmiller Where newer = more descendant. If there is no descendant relationship (independent branches) then it’s an error #2019-01-15 15:17:19 alexmiller (same as we do for Maven artifacts) #2019-01-15 20:37:59 avi So in this library I’m hoping to help out with, I’ve moved the dependencies from project.clj to deps.edn and added the plugin lein-tools-deps to project.clj. lein deps seems to work just fine, but when I run lein test I get:
$ lein test
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
Tests failed.
I tried changing the clojure version specified in deps.edn to 1.9.0 and 1.10.0 but that seemed to have no effect. Anyone know what’s happening here, what I’m missing? #2019-01-15 20:49:41 ghadi seems like your tests are not picking up the dependencies #2019-01-15 20:50:56 ghadi I wonder if your lein-tools-deps plugin isn't working in the test profile #2019-01-15 20:57:08 avi Hmm, OK, I’ll look into that — thanks! #2019-01-15 21:07:34 avi Aha, I needed to add :middleware and :lein-tools-deps/config to project.clj — as explained clearly in the docs of lein-tools-deps! (docs ftw) #2019-01-15 21:07:44 thosmos @aviflax I had that same problem with Cursive and the only thing I found to get around it was to downgrade lein-tools-deps to "0.4.1" #2019-01-15 21:09:55 avi Have you tried adding those keys to your project.clj ? #2019-01-15 23:42:10 aaelony I have a codebase that I’ve instrumented with tools-deps and can run via clj -m something.main <args>
from the command line. What do folks typically do or what guidance is out there to make it easier for other (non clj) users to run it. In the past, I’d have done a lein uberjar and wrapped it in a bash script. Is there a better way? #2019-01-16 03:14:19 souenzzo About depstar/tools-deps jar'ing:
I need to deliver a jar that will run with java -jar my.jar. It's a client requirement.
There is some way to create it using tools-deps eco-system??
PS: without use things like clj -A:lein -m lein.uberjar #2019-01-16 18:41:15 dominicm Pack works differently to leiningen, intentionally. #2019-01-16 18:51:27 seancorfield I started to look at adding manifest support to depstar and realized I'd have to deal with AOT/`compile` unless you just set Main-Class: in the manifest to clojure.main... and I didn't have any enthusiasm for fixing that part 🙂 I may take another run at that at some point (it seems like it needs to create a temp folder, add it to the classpath, bind *compile-path* to that temp folder path, and run clojure.core/compile for the named main namespace … which just seems like a lot of work, just to enable java -jar and not need -m entry.point!). #2019-01-16 18:55:23 dominicm @seancorfield can depstar take a flag to add a folder? That's how pack makes AOT work #2019-01-16 18:56:30 seancorfield depstar deliberately takes no options at the moment. It's "simple", rather than "easy" 🙂 #2019-01-17 00:31:11 souenzzo Maybe someone will make this simple thing easy... 😜 #2019-01-16 01:26:49 ajoberstar @alexmiller Still looking for community contributions on the Windows clj script? I'm interested and have PowerShell experience.
(Sorry for the dupe if you already saw my previous post and just haven't had a chance to reply.) #2019-01-16 01:44:30 alexmiller I saw it, thx, just been busy on other stuff #2019-01-16 03:27:56 seancorfield @souenzzo yes some of the tools.deps libraries add a manifest when building an uberjar. I had it in depstar at one point but preferred fewer options. #2019-01-16 03:29:19 seancorfield Open an issue and I'll put that code back in. Or you can try the other tools. I'm on my phone so I can only point you at the wiki in the tools.deps repo for a list. #2019-01-16 06:19:59 seancorfield (It would help if Issues were enabled for my project I guess… fixed!) #2019-01-16 18:42:23 dominicm @seancorfield any advice for running clj on WSL? Gotchas to be aware of? Preparing for a training course, I seem to have edge working, but I'm expecting real life issues. #2019-01-16 18:44:53 seancorfield The only annoying thing I've run into is that if you edit code on Windows, connected to a REPL of any sort running on WSL, any file-level ops you attempt will fail due to the mismatch of filepaths between the two environments. For example, a hot key to "load file" in your editor will generally send a full file path to the REPL, which in this case will start with C:\… when the WSL REPL is expecting /… #2019-01-16 18:45:25 seancorfield My workaround for that is select-all, eval-selection (but I may try to "fix" it in Chlorine directly). #2019-01-16 18:46:26 seancorfield Oh, caveat for that: I use a project repo path on Windows of something like C:\projects\myrepo and have /projects symlinks on WSL to /mnt/c/projects so the file paths are almost the same. #2019-01-16 18:47:01 seancorfield (that way, just dropping C: and using / instead of \ will work on both sides -- modulo the editor file path issue) #2019-01-16 18:48:08 seancorfield Also, since WSL is headless, you can't run REBL on it (so you end up having to hack that together with Leiningen on the Windows side. #2019-01-16 18:48:24 seancorfield (I haven't made any serious attempt to try to debug/workaround that) #2019-01-16 18:49:42 dominicm That's a useful tip, thanks! #2019-01-18 21:42:28 Christian Johansen is there a special trick to deploying ClojureScript libraries to clojars with tools.deps? I tried doing clojure -Spom and then mvn deploy. I certainly did put a jar in clojars, but trying to use it I discover that it doesn't contain any sources 🤔 #2019-01-18 22:24:21 Christian Johansen Thanks! #2019-01-18 22:24:31 Christian Johansen Seems my mistake was not building the jar, D'OH! #2019-01-18 22:24:46 Christian Johansen thought it would be auto-built. and it was, but the jar maven built was useless, unsurprisingly, as it doesn't know about cljs files I guess #2019-01-18 22:25:54 Christian Johansen used mvn deploy:deploy-file with a prepackaged jar instead #2019-01-19 14:19:47 Christian Johansen I've had various minor issues figuring out how to combine tools.deps, figwheel-main, devcards, emacs/CIDER and Clojars, and got a lot of help in here, and now I've summarized my experience in a howto on configuring this combination: https://cjohansen.no/tools-deps-figwheel-main-devcards-emacs/ #2019-01-19 21:01:17 richiardiandrea thanks this is nice #2019-01-19 21:02:56 richiardiandrea the only difference I have there is deploying with deps-deploy...also sooner or later I would like to be able to sign the jar. #2019-01-19 14:19:54 Christian Johansen hope it can be of help to someone #2019-01-19 21:00:59 richiardiandrea @christian767 I have recently added deps.edn-only deploy in https://github.com/elasticpath/fonda - actually taking inspiration from @eval2020's work #2019-01-19 21:03:19 richiardiandrea sorry I read I am late...too late 😄 #2019-01-19 21:28:49 Christian Johansen Yeah @slipset tipped me about deps-deploy on Twitter, I'm going to update the article to recommend it in place of maven :+1: #2019-01-19 21:35:14 slipset @richiardiandrea jar signing just landed on master for deps-deploy. Comments welcome. #2019-01-19 21:36:20 Christian Johansen 👌 #2019-01-19 21:37:54 slipset Had to read a bit too much to figure out how to do that. #2019-01-19 21:39:32 slipset The lein code for that is somewhat b0rken (reading the gpg passphrase is a bit unstable at times), so I had to reimplement that… #2019-01-19 21:41:02 Christian Johansen Ugh, gpg. Can't think of anything that makes me feel more like a moron #2019-01-19 21:42:24 richiardiandrea Wow timely! Thank you! #2019-01-19 21:43:29 slipset Inspired by @christian767’s blog 🙂 #2019-01-19 21:45:31 Christian Johansen 😊 #2019-01-19 21:55:31 richiardiandrea That blog is really comprehensive, I am thinking about making it a template now 😺 #2019-01-19 22:00:46 Christian Johansen hehe, please do. I considered making it a minimal github repo, but time didn't allow for it between shipping kids around, walking the dog and putting together IKEA furniture :shrug: 🙂 #2019-01-23 17:23:30 rschmukler I noticed a branch (`add-lib`) on the github repo, but it's ~20 commits behind master. Is it still preferable to use it? #2019-01-23 18:15:53 alexmiller we haven’t integrated it yet #2019-01-23 18:16:14 alexmiller I can merge up to master though if you want a master-compatible version of the branch #2019-01-23 18:17:14 alexmiller actually I did that recently, and add-lib branch is currently only like 1 commits behind master #2019-01-23 18:22:31 alexmiller merged, you can use sha e160f184f051f120014244679831a9bccb37c9de for latest #2019-01-23 18:43:07 rschmukler @alexmiller Thanks a ton! Heads up that I found a bug where a transient dependency wasn't reloaded into the classpath. I will see if it's fixed by the latest, and if not, open an issue. #2019-01-23 18:43:35 rschmukler Unless you'd prefer me not opening issues for it, since it's not even part of a release yet #2019-01-23 19:48:36 alexmiller Did you mean transient or transitive? #2019-01-23 19:49:13 alexmiller add-lib won’t do re-loading - it’s for additive use #2019-01-23 21:36:10 ghadi @seancorfield have you run into any issues with depstar and multi release jars? #2019-01-23 21:38:34 seancorfield @ghadi What do you mean by “multi release”? #2019-01-23 21:39:48 alexmiller as of java 9 you can make jars with different per-jdk sets of stuff in them #2019-01-23 21:40:45 seancorfield I haven’t tried to do anything with Java 9+ yet — we have a bunch of legacy code that won’t run on it. #2019-01-23 21:42:01 ghadi I hadn't considered MRJars and was curious if it panned out #2019-01-23 21:44:07 seancorfield Given that depstar doesn’t add a manifest — and filters out some manifest-related stuff as I recall — I don’t know how it would even go about that. #2019-01-23 21:45:37 ghadi I forgot whether it filtered anything or not from the Meta-Inf #2019-01-23 21:45:42 seancorfield I started looking at adding basic manifest support but that seemed to lead me down the AOT path, at least for the Main-Class: entry when it wasn’t clojure.main … and I decided I didn’t have the stomach for that (yet). #2019-01-23 21:45:46 ghadi If it works it's by accident #2019-01-23 21:46:48 seancorfield (defn excluded?
[filename]
(or (#{"project.clj"
"LICENSE"
"COPYRIGHT"} filename)
(re-matches #"(?i)META-INF/.*\.(?:MF|SF|RSA|DSA)" filename)
(re-matches #"(?i)META-INF/(?:INDEX\.LIST|DEPENDENCIES|NOTICE|LICENSE)(?:\.txt)?" filename)))
#2019-01-23 21:47:45 seancorfield (which seems pretty arbitrary — and I haven’t rationalized why it excludes all of that lot) #2019-01-23 22:21:26 dominicm Does anyone have a firm reason that pack should implement a depstar like implementation? #2019-01-23 22:21:45 ghadi what does that mean? #2019-01-23 22:26:50 dominicm Rather, is there a reason pack should adopt "jar Smashing"? #2019-01-23 22:31:33 hiredman I've got a thing I worked for a little bit, but I don't think I have the time to go anywhere with it: https://gist.github.com/hiredman/d68cafb6aa8cea563c7b77d54f522421 instead of building uberjars (which can end up taking forever to transfer) it uploads individual jars/git repos/local deps only if they haven't already been uploaded and then deploys a file containing a built classpath pointing to those things #2019-01-23 22:32:20 hiredman (inspired by a talk rich gave somewhere about datomic ions) #2019-01-24 07:22:08 dominicm That's really neat. #2019-01-24 07:54:19 plexus has anyone already come up with a thing that reloads deps.edn, adding any newly added libraries to the classpath? #2019-01-24 08:19:26 dominicm Alex's add-lib would probably be pliable to that. If you build it, I would be interested. #2019-01-24 09:36:41 thheller FWIW I have done a few experiments with dynamically adding dependencies at runtime for shadow-cljs #2019-01-24 09:37:17 thheller the only case where this works reliably is when adding dependencies that don't depend on anything different than what is already loaded #2019-01-24 09:37:27 thheller as soon as there are any version conflicts things get icky #2019-01-24 09:38:14 thheller eg. lib-a depends on X version 1.0.0 and is loaded. lib-b wants to be added but would cause X version 1.2.0 to be loaded instead of 1.0.0 if constructed normally #2019-01-24 09:38:38 thheller since we can't easily unload version 1.0.0 you might now not be able to load properly #2019-01-24 09:39:04 thheller unfortunately that scenario is quite common for bigger libs #2019-01-24 09:39:13 thheller works well enough for simple libs without deps though #2019-01-24 09:39:50 thheller gets even more complicated when accounting for npm deps 😉 #2019-01-24 09:39:56 plexus I'm guessing pomegranate/alembic and friends also have this issue then #2019-01-24 09:40:23 thheller deps resolves version conflicts slightly differently so it is more prone to this #2019-01-24 09:40:30 thheller but yes same issue is present everywhere #2019-01-24 09:40:31 plexus I see #2019-01-25 16:26:09 alexmiller I don’t think deps is any “more prone” to this than pomegranate. it’s a fundamental issue with how libraries and classes are loaded on the jvm. #2019-01-25 16:26:48 alexmiller if you’re interested in clj on Windows, I have created #clj-on-windows as a working channel for anyone that wants to help with it #2019-01-26 01:38:09 alexmiller thx, I prob won’t have time till next Friday #2019-01-30 01:21:12 lilactown is there a way to just get whatevers at master with git deps? #2019-01-30 01:21:25 lilactown try to write a quick-try command for a lib #2019-01-30 02:34:41 alexmiller No #2019-01-30 02:35:16 alexmiller By design, we want to use stable versions for caching purposes #2019-01-30 02:35:41 alexmiller But you can use the gitlibs lib to look up what sha master is #2019-01-30 02:43:07 alexmiller you can even one-line it - clj comes with a built-in :deps alias that includes tools.deps (and tools.gitlib): #2019-01-30 02:43:13 alexmiller $ clj -A:deps -e "(require 'clojure.tools.gitlibs)" -e '(println (clojure.tools.gitlibs/resolve "" "master"))'
d86fec647b705b5d01d27c32322c9ccc9f65b77c
$ clj -Sdeps '{:deps {org.clojure/tools.gitlibs {:git/url "" :sha "d86fec647b705b5d01d27c32322c9ccc9f65b77c"}}}'
Checking out: at d86fec647b705b5d01d27c32322c9ccc9f65b77c
Clojure 1.10.0
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/conform (s/spec int?) 10)
10 #2019-01-30 02:48:26 alexmiller rub some scripting on that and you’ve got something #2019-01-30 03:45:23 seancorfield Could you combine that with the add-lib branch and dynamically load the master version into your REPL? (I can't remember whether add-lib works with git libs) #2019-01-30 03:56:58 alexmiller sure #2019-01-30 03:57:27 alexmiller add-lib works with any dep supported by tools.deps #2019-01-30 04:07:21 seancorfield Yeah, nice user=> (require '[clojure.tools.deps.alpha.repl])
nil
user=> (require '[clojure.tools.gitlibs])
nil
user=> (defn load-master [lib] (let [git (str "" lib ".git")] (clojure.tools.deps.alpha.repl/add-lib lib {:git/url git :sha (clojure.tools.gitlibs/resolve git "master")})))
#'user/load-master
user=> (load-master 'clojure/tools.trace)
Checking out: at fbba1a5012bae0ac4ed5cd704babf23e9e3fbc44
Downloading: org/clojure/clojure/1.4.0/clojure-1.4.0.pom from
Downloading: org/clojure/clojure/1.4.0/clojure-1.4.0.jar from
true
user=> #2019-01-30 04:11:29 seancorfield ^ @lilactown #2019-01-30 04:13:39 lilactown Clever! Thanks 😁 #2019-01-30 04:46:20 seancorfield I ❤️ Clojure 🙂 #2019-01-30 04:47:07 seancorfield (interesting that clojure/tools.trace has a dependency on Clojure 1.4.0!) #2019-01-30 04:56:16 alexmiller a lot of the contribs depend on the oldest version they can support #2019-01-30 04:56:45 alexmiller CI no longer tests anything older than 1.7 iirc #2019-01-30 17:19:21 jvtrigueros I haven't taken the plunge into using tools.dep yet, but absorbing via osmosis through Sean's code snippets! Thanks! #2019-01-31 21:01:27 borkdude is it possible to access environment variables in deps.edn like $HOME? probably not.. #2019-01-31 21:01:52 dominicm Nope! Why do you ask? #2019-01-31 21:02:34 borkdude because I want to say ~/git/... in a local dep. just a convenience #2019-01-31 21:04:28 dottedmag Is it a transient thing for working on a local copy of a library? clj -Sdeps "...$HOME..."? #2019-01-31 21:05:46 borkdude yup, can do it like that #2019-01-31 21:55:43 unbalanced hi all... so I've been attempting to transition over to the deps.edn -- first of all, is there a name for this style of build tools? aka lein vs boot vs ... what... deps?. In general it's great but there seems to be "something else going on". I'm seeing blog posts about a build.clj file but I don't see anything official on it. Can anyone explain what that's all about or have a good reference for it? #2019-01-31 22:02:54 lilactown @goomba leiningen and boot are fully-featured toolbelts for doing things like fetching and loading dependencies, building uber jars, running tests, etc. #2019-01-31 22:03:58 unbalanced nod #2019-01-31 22:04:22 lilactown tools.deps is only about fetching and dependencies and building a classpath to run your JVM program. it doesn't handle any of the other things #2019-01-31 22:04:54 unbalanced okay, maybe I'm in the wrong place? is the clj tool not part of this and deps.edn? #2019-01-31 22:05:00 alexmiller it is #2019-01-31 22:05:17 alexmiller clj is about two things: making classpaths from transitive deps, and launching programs #2019-01-31 22:05:34 alexmiller deps.edn describes the deps and ways to modify the making of classpaths #2019-01-31 22:05:54 alexmiller all other tasks are programs, so you launch a program to do the task #2019-01-31 22:05:56 unbalanced so far it's been great at those 😄 #2019-01-31 22:06:14 unbalanced is that where these build.clj files I'm seeing come in? #2019-01-31 22:06:16 alexmiller ^^ some tools #2019-01-31 22:06:20 alexmiller no one idea what build.clj is #2019-01-31 22:06:34 alexmiller but it’s not deps.edn / clj #2019-01-31 22:07:15 lilactown because clj doesn't do all those other things, some people have decided to write their own script for e.g. building a JAR file, or bundling the CLJS code. instead of using a leiningen feature or plugin #2019-01-31 22:07:17 unbalanced here's the only example I can find #2019-01-31 22:07:30 lilactown one might decide to call that script build.clj 🙂 #2019-01-31 22:08:09 unbalanced okay so that stuff is essentially home-rolled scripting that some decide to call build.clj #2019-01-31 22:08:54 lilactown yeah. in this case, they wrote a script to bundle their CLJS code for deploying #2019-01-31 22:09:01 unbalanced interesting. That is the one area in which I'm hurting right now -- I've been running lein in parallel with clj because I can't find a reliable way to do uberjars #2019-01-31 22:09:21 lilactown in leiningen, I believe there's a cljsbuild plugin to do that for you #2019-01-31 22:09:28 lilactown ah yep. same for uberjars #2019-01-31 22:09:58 unbalanced mhm. Are uberjars not really a thing for clojure? I assumed that was the defacto standard for deployment #2019-01-31 22:10:49 alexmiller they are very common. there are at least 3 tools in that tools page above that build them #2019-01-31 22:12:22 unbalanced indeed. Just trying to consolidate methodologies as much as possible. #2019-01-31 22:13:05 unbalanced thanks everyone for fielding questions 😄 #2019-01-31 22:13:23 lilactown I think some people are moving away from them for Clojure, since there's not a ton of benefit you get other than improved startup time in most cases #2019-01-31 22:14:09 alexmiller they are certainly convenient as a unit of deployment #2019-01-31 22:16:23 unbalanced what do they otherwise deploy with then? clj -m mything.core? #2019-01-31 22:20:37 lilactown ah I was also conflating it with AOT. but currently I throw it all in a docker container because I already have infrastructure for that ¯\(ツ)/¯ #2019-01-31 22:21:24 alexmiller AOT is orthogonal #2019-01-31 22:59:56 seancorfield @goomba FWIW, at work we use (my fork of) depstar to build all-source uberjars for pure convenience of shipping an "artifact" to a server; to run those we do java -cp path/to/the.jar clojure.main -m entry.point ... where entry.point is the namespace containing the -main we wish to run (our artifacts often contain several, related -main functions in different namespaces). #2019-01-31 23:00:48 seancorfield All of our dev/test process uses clojure (the core of clj) wrapped in a small bash script to simplify handling of command line options. #2019-01-31 23:00:53 unbalanced gotcha (worth a lot, as always!) #2019-01-31 23:01:06 seancorfield We do not AOT anything. #2019-01-31 23:03:27 seancorfield It's also worth mentioning, as background, that we started with lein back in 2011 (ish -- that was when we first took Clojure to production), and we switched completely to boot at the end of 2015 (I think, I'd have to go consult my blog to be sure), and then we switched completely to clj/`deps.edn` last year. #2019-02-01 13:01:20 misha is there good bare-bones template for clojure+clojurescript monorepo with tools.deps? #2019-02-01 18:03:15 dominicm Edge isn't quite bare bones, but is the only thing I know of without having looked much #2019-02-02 00:21:15 misha reading through the docs now, thanks #2019-02-01 13:14:00 misha {:deps {com.bhauman/figwheel-main {:mvn/version "0.2.0"}}
opieop #2019-02-01 19:53:21 eraad Hi! I’m trying to use this library (originally built with Leiningen) from a clojure-cli project:
https://github.com/datil/xmldsig
I added a deps.edn but I have a couple of questions:
1. The lib has a src/main/java source folder. Do I need to add this as a “src” folder in the deps.edn?
2. I’m trying to import it using git deps, it adds to the classpath but I cannot seem to be able to import it.
Any recommendations on where to look? #2019-02-01 19:54:31 hiredman you won't be able to use it #2019-02-01 19:54:39 alexmiller libraries with Java source can’t be used this way #2019-02-01 19:54:51 alexmiller clj is designed for libs with no “build” step #2019-02-02 06:36:48 matthias This is going to be game changing too - we're applying the idea in some really interesting ways. #2019-02-02 06:42:59 dominicm Can you share more about how you're able to leverage this to great effect? #2019-02-01 19:55:07 alexmiller which is most Clojure libraries, but not those with Java source #2019-02-01 19:55:40 alexmiller you can use the built version as an artifact via Maven or via a locally built jar by pointing to it with a :local/root coordinate #2019-02-01 20:07:03 eraad Cool @alexmiller @hiredman thanks! #2019-02-01 21:50:22 viesti if gen-class would work at the repl, wouldn’t this mean a build-free support of libs using gen-class? 🙂 #2019-02-01 21:50:30 viesti don’t know how many those would be though #2019-02-01 21:56:49 dominicm @viesti you have conflicting files in the class path, nothing can save you now. The ordering is pretty much random, oz is working by coincidence. #2019-02-01 21:57:32 dominicm Fwiw, you could resolve this if you are a bit crazy by iterating over all of the matches. #2019-02-01 21:57:41 viesti yeah, the moment when you realise that the floor starts moving #2019-02-01 21:58:23 dominicm I'd recommend that public/index.html isn't used, and that a namespaced prefix is used instead. #2019-02-01 21:59:15 viesti hum, that might be it #2019-02-01 21:59:33 viesti since might be difficult to fish out the conflicting dependency #2019-02-01 22:01:24 dominicm The other library, json-to-html should also not use public/index.html #2019-02-01 22:04:05 viesti yeah, could point this out to the other lib #2019-02-01 22:04:25 viesti makes me think on using public/index.html in my own apps… 🙂 #2019-02-01 22:26:59 dominicm You should only use it there ! #2019-02-01 22:30:12 viesti yeah, should report to the other dep actually, I think #2019-02-01 22:34:20 viesti seems that markdown-to-hiccup-0.6.1.jar might have unnecessary resources in it’s artifact #2019-02-01 22:35:13 dominicm It's both really. Unless oz is an app. #2019-02-01 22:35:13 viesti spend the day pulling a hairball of Ansible dependencies, now wondering how I ended up to another ball of strings in the evening #2019-02-01 22:35:32 viesti thinking I’m good at catching balls of strings #2019-02-02 06:43:55 dominicm I've only learned this stuff from getting tangled too #2019-02-02 06:44:09 dominicm Some of us are destined for tangling #2019-02-02 06:46:39 dominicm now figwheel main doesn't work... #2019-02-02 07:34:06 viesti The case of the creeping index.html... Trying to hold off thoughts of hijacking webapps by malicious classpath resource injection 🙀 #2019-02-02 07:48:10 dominicm a lot of people unknowingly serve a /license.html as part of jetty iirc #2019-02-02 07:48:36 dominicm oh, no, it's an about.html #2019-02-02 07:48:44 dominicm I found out when I discovered the page on our site #2019-02-02 10:40:09 viesti heh, there really is such a resource, didn’t find it from served from couple of our apps #2019-02-02 16:04:28 dominicm Depends how you set it up, some setups serve html from the root :) #2019-02-02 17:57:08 viesti yup, duct template has nice default, which creates a folder with the app name for the resources #2019-02-01 22:37:20 viesti probably both, although oz is higher in the hierarchy, but it can be a dependency itself #2019-02-01 23:22:22 viesti almost forgot, thanks for the advice! :) #2019-02-04 11:33:06 borkdude In a mono-repo is it possible to use a common deps.edn file with aliases that can be used from all sub repos? #2019-02-04 11:33:34 borkdude (without hacking $CLJ_CONFIG) #2019-02-04 12:11:32 dominicm Not that I've discovered. #2019-02-04 12:11:39 dominicm I don't believe it's possible. #2019-02-04 12:24:40 alexmiller No, not currently #2019-02-04 12:30:18 borkdude The idea is that you can define a bunch (bundle) of deps in a common file and then refer to a common name for that bundle in your deps.edn files’ :deps and/or :extra-deps. #2019-02-04 13:29:04 borkdude thanks #2019-02-04 17:53:25 seancorfield We have a monorepo with dozens of subprojects -- and we use a small shell script to wrap clojure and use CLJ_CONFIG to point to the monorepo's deps.edn file (and then each subproject has its own deps.edn file). Works great. Any reason you don't want to do that @borkdude? #2019-02-04 17:53:59 seancorfield Basically the shell script does cd path/to/subproject.clj; CLJ_CONFIG=../versions/deps.edn clojure ...args... #2019-02-04 18:12:02 richiardiandrea datapoint -> we have a monorepo and looking for a clean solution as well #2019-02-04 18:12:28 richiardiandrea or hints at how to contribute a solution #2019-02-04 18:20:50 borkdude @seancorfield the reason I thought I wanted to avoid that is that it messes with people’s own configs in their .clojure dir? #2019-02-04 18:24:22 alexmiller some possible answers, curious if these would help:
1) a clj flag to pass one or more config files that go “in the chain” #2019-02-04 18:24:44 alexmiller 2) an env property to list or more additional config file locations #2019-02-04 18:25:35 alexmiller 3) some kind of config file in .clojure/ (I’m guessing probably not ideal as this is for a group of projects but may not extend to everything you’re doing) #2019-02-04 18:37:41 dominicm 1 makes this manual opt in everywhere, so users have to inform their tooling of this flag.
2 would require something like direnv, and would also require extensive editor integration. #2019-02-04 18:38:26 borkdude another way: :include "../parent.deps.edn" #2019-02-04 18:38:29 dominicm 1 seems more tenable than 2 for something like cursive which has the most extensive analysis of deps.end files. #2019-02-04 18:39:57 borkdude That, together with a way of including the deps of aliases into another alias would be helpful to me #2019-02-04 18:40:00 ghosttoaster Hey all. I'm having trouble getting a very simple deps.edn going. I keep on getting a message that opencv can't be found in mvn central, but I've added my custom repo where the jar can be found. #2019-02-04 18:41:36 eval2020 1 with less magic: clj --deps-file foo/deps.edn would only take foo/deps.edn into account. The current situation could be seen as clj --deps-file ~/.clojure/deps.edn --deps-file deps.edn .... #2019-02-04 18:42:05 seancorfield @borkdude when you're working on a monorepo project, I'd argue you want a more reproducible set of processes so I'd say you want .clojure to be omitted from deps.edn. #2019-02-04 18:42:31 dominicm @seancorfield I don't think that's any more true than a normal repo is it? #2019-02-04 18:42:33 borkdude @seancorfield people tend to put their cider-nrepl config into .clojure/deps.edn. I don’t want to disturb that #2019-02-04 18:43:35 borkdude Since deps.edn is “just data”™ I’m inclined to rewrite a template to the deps.edn that will be used #2019-02-04 18:43:46 dominicm I also put Rebel, library overrides for environment specific bugs, benchmarking and more. All depending on the context I'm in. #2019-02-04 18:44:33 seancorfield @dominicm It's a matter of opinion 🙂 We've generally added stuff to our versions/deps.edn file that individual devs find useful. And we have a couple of "dev" namespaces that include REPL startup stuff. So there's really nothing in our .clojure/deps.edn files that would be useful in addition (we already have nrepl/socket REPL/REBL stuff in our project deps.edn files). #2019-02-04 18:44:41 dominicm @borkdude be aware that updating a deps.edn like that will erase comments. I wrote ednup to be a crappy solution to that, but it's not prime ready yet. #2019-02-04 18:45:09 borkdude @dominicm I’m exploring rewrite-clj using zippers, that will leave everything intact. https://github.com/Olical/depot does this too. #2019-02-04 18:45:25 seancorfield That said, I wouldn't object to an enhancement to tools.deps/`clj` that allows additional deps.edn files to be specified 🙂 #2019-02-04 18:45:26 dominicm @borkdude that's what ednup does #2019-02-04 18:46:10 borkdude :extra-deps.edns #2019-02-04 18:46:56 dominicm @seancorfield I work on a few monorepos (they're mono to their context), and updating my dev config in each of them would be a pain. I'd feel inclined not to commit my dev hacking and clutter git history, especially in projects where we have stringent reviews on code. #2019-02-04 18:47:20 dominicm @borkdude yup. You can see usage in my rewrite of resolve tags. #2019-02-04 18:48:55 dominicm It's not a great library, but I plan to iterate on it intensely at some point, and it's great for simple things. #2019-02-04 18:49:15 dominicm Oh, if you want examples, pack has an injector which uses rewrite directly. #2019-02-04 18:51:06 borkdude I meant: figuring out the format/API is hard, not the actual programming 🙂 #2019-02-04 18:51:41 borkdude I want to support existing users of boot-bundle (including myself) and support a migration path from boot to deps.edn #2019-02-04 18:52:21 borkdude anyways, being able to specify multiple extra deps.edns from the cmd-line is a good thing in my opinion #2019-02-04 18:52:56 borkdude you can always use -Srepro to exclude .clojure so that functionality is already there #2019-02-04 19:02:46 alexmiller it is still a pending todo to finalize what -Srepro means, esp in context of -Sdeps and other features. some of that is pending config file rework #2019-02-04 18:53:31 hiredman another option, instead of having a super deps.edn for the monorepo, or generating dep.edns for subprojects might be something like a dependency linter, you give it some rules (deps on project x most version y) and run it over a directory and it checks all the deps.edn files it finds #2019-02-04 18:54:08 hiredman a linter could even be run against projects not in your monorepo to see how some other library might integrate #2019-02-04 18:54:17 borkdude @hiredman meh, then you would still have to update all those occurrences, or do you mean the linter would actually update the files? #2019-02-04 18:54:38 hiredman no, you would have to update the files #2019-02-04 18:55:05 borkdude the reason I wrote boot-bundle was to prevent having to do that, it became annoying #2019-02-04 18:55:35 borkdude I would want one source of the truth and the include that in another project #2019-02-04 18:56:18 borkdude some call it “managed dependencies” but I didn’t realize that when I wrote the lib #2019-02-04 18:56:43 hiredman how much dependency churn do you have? #2019-02-04 18:57:22 borkdude so what I’m searching for is managed deps for deps.edn. Another way to accomplish this maybe is to have “empty” projects that declare their deps and then make a dependency on those with local/root. #2019-02-04 18:58:02 borkdude but that gets out of hand too, I want it to be declarative and easy to modify #2019-02-04 18:59:06 borkdude @hiredman we have a couple of local libraries which all depend on subsets of the same dependencies. in some of those we use AOT. it’s nasty if you get the versions wrong, so we want them to be the same #2019-02-04 18:59:32 hiredman sure #2019-02-04 19:03:18 alexmiller we always intended to support something like managed deps and I think that is a valuable use case #2019-02-04 19:04:06 alexmiller we have not put as much thought into how that should look as with other stuff yet #2019-02-04 19:04:47 seancorfield Our versions/deps.edn file is mostly :override-deps to "pin" dependencies for the whole monorepo. Each subproject declares its own dependencies as group/artifact {}. And we have dev/test/build aliases in versions/deps.edn. #2019-02-04 19:05:29 borkdude @alexmiller with being able to specify extra deps.edn configs from the cmd line + the ability to merge multiple aliases into one (:foo has :bar and :baz like with -R:bar:baz) I think we would be 99% there #2019-02-04 19:05:36 borkdude maybe even 100% #2019-02-04 19:05:40 seancorfield Having four deps.edn in the chain instead of three would solve all our issues 🙂 (system, user .clojure, project root, subproject root). #2019-02-04 19:07:10 borkdude project/subproject seems arbitrary to me (different projects use different directory structures), being able to specify this yourself and number of times you can do this should be flexible #2019-02-04 19:08:33 hiredman aot adds whole other wrinkle on top of things for monorepos, because you are depending on build output of a local project, not the local project (with clj doesn't directly support so much) #2019-02-04 19:16:11 hiredman do you aot as low down the deps tree as possible, or as high up as possible? #2019-02-04 19:17:46 borkdude @hiredman We wrote a lib that has some Java code which we compile up front. We don’t want to have the situation that we’re going to use different deps than what that was compiled against. #2019-02-04 19:18:27 hiredman ah interesting, built java not aot'ed clojure? #2019-02-04 19:18:30 seancorfield @borkdude Perhaps I'll rephrase that as "monorepo" and "project" level then instead of "project" and "subproject"? 🙂 #2019-02-04 19:19:03 borkdude We also ran into trouble once with a service that was running clojure 1.8 which did different edn serialization than 1.9, that was also a problem. we just don’t want to think about this stuff too much and exclude any such possibility. #2019-02-04 19:20:11 borkdude @seancorfield boot-bundle supports loading bundles (managed deps) even from Clojars, it doesn’t matter how you organize this. #2019-02-04 19:21:42 borkdude @hiredman yes #2019-02-04 19:28:25 seancorfield @borkdude and each "bundle" would map to a separate deps.edn file? So you could have a folder for each "bundle" and then access them via multiple :local/root deps? #2019-02-04 19:29:18 borkdude @seancorfield how this is done currently in boot bundle is one keyword maps to one or more deps (or recursively, to another bundle). this is just EDN which is expanded dynamically in the scope of a build.boot file. #2019-02-04 19:29:26 seancorfield For us, the problem isn't really the deps themselves, but the aliases we want across the whole monorepo -- sort of like requiring every dev to have consistent .clojure/deps.edn files but just for the projects in the monorepo. #2019-02-04 19:29:34 borkdude where this bundle.edn file comes from is up to you, could be from clojars, a directory, etc. #2019-02-04 19:30:22 seancorfield I haven't looked at anything in Boot for quite a while now. I hadn't heard of boot-bundle. Sounds useful in a deps.edn world. #2019-02-04 19:32:12 borkdude in boot this is easy to implement because you can actually program in the build file. in deps.edn I currently see no other way than to rewrite a deps.edn.template and expand it into a deps.edn. but with the options we mentioned in this conversation, I think it’s very close at hand #2019-02-04 19:33:00 borkdude relying on add-lib could also work, but I’m not too familiar with that and I think that’s alpha-alpha? #2019-02-04 19:36:08 alexmiller alpha-omega #2019-02-04 19:42:09 seancorfield I recently added some aliases to my .clojure/deps.edn file that pull in that branch and the comments in my EDN file show how to write a load-master function that can resolve master from git repos and then add that library 🙂 #2019-02-04 23:57:17 richiardiandrea read the whole conversation, our team is looking for the same answers @borkdude, maybe we can join forces 😄 Share the pain err your thoughts with me ok? #2019-02-05 05:05:15 borkdude @richiardiandrea Totally! #2019-02-05 08:22:51 Christian Johansen >Error building classpath. Manifest type not detected when finding deps for myapp/myapp in coordinate #:local{:root "/Users/christian/projects/myapp"} #2019-02-05 08:23:32 Christian Johansen I'm getting this on a project that used to work fine. The local dependency is a leiningen based project. Am I doing something wrong? #2019-02-05 08:29:48 Christian Johansen hmm, maybe I used to have a deps.edn file in that project? 🤔 #2019-02-05 12:57:11 alexmiller Probably. You can force it with :deps/manifest :deps (and it’s tolerant of a missing deps.edn) #2019-02-05 13:05:07 Christian Johansen Ok, good to know! Solved it by extracting project.clj deps into deps.edn. This is work in progress, and I've had a deps.edn file that wasn't commited to git that I probably git reset away 🙈 #2019-02-05 10:19:53 dominicm Is there a jira ticket for managed deps? #2019-02-05 10:42:35 borkdude I’m willing to post my proposal into a JIRA ticket, but I thought it would be good to have a little screening first #2019-02-05 10:44:40 dominicm Sure :) I actually wanted somewhere to direct the above issue to. #2019-02-06 18:08:13 timgilbert Is there anything like an offline mode for tools.deps? I'm finding my CI builds take 2-2.5 minutes per alias per build step just downloading maven-metadata.xml files from my S3 repo and clojars / maven-central, and I can't seem to cache the results #2019-02-06 18:09:12 timgilbert This is separate from the actual .jar / .pom files which I'm able to cache with no problem (in ~/.m2) #2019-02-06 18:13:50 borkdude We use s3 wagon private in boot and I notice it downloads metadata more often than it should, which also slows down things there #2019-02-06 18:14:31 timgilbert Also, these specific packages have become my arch-enemies as every time I run clojure I see this block of text:
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-dynamodb/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-kinesis/maven-metadata.xml from
Downloading: com/amazonaws/aws-java-sdk-kinesis/maven-metadata.xml from
#2019-02-06 18:16:00 timgilbert I don't know if there's a TTL set to a ridiculously low value by AWS or what, but I just cannot avoid seeing that every single time if I've got those dpes #2019-02-06 18:24:48 alexmiller file a ticket with as much info as possible #2019-02-06 18:25:43 avi I suspect the answer is no, but just to confirm: is there any way to have tools.deps compile Java code when consuming a Clojure library via git that also includes some Java code? #2019-02-06 18:27:30 alexmiller no #2019-02-06 18:27:44 avi Alrighty then, thanks! #2019-02-06 23:03:49 kenny Is there a way to see why a classpath refresh takes a long time with clj? #2019-02-06 23:08:28 alexmiller there is a super secret verbose knob you could try #2019-02-06 23:09:03 alexmiller clj -Sdeps '{:aliases {:v {:verbose true}}}' -A:v #2019-02-06 23:10:19 alexmiller the most likely culprit is slow I/O of course, which you can really spot only by observing where that pauses #2019-02-06 23:10:55 kenny The weird part is it only occurs in one particular project. #2019-02-06 23:11:09 alexmiller the next most likely is that tools.deps is doing more work than it should, which you may also be able to spot by seeing it redoing work #2019-02-06 23:14:14 kenny Hard to tell what's expected and whats not. Expanding git deps takes a good 3-5s for this project. #2019-02-06 23:16:44 kenny This project also has a fairly large dependency tree. The ~20s refresh gets a bit painful though. #2019-02-06 23:19:12 alexmiller git deps are stored in two parts - the git objects, and the working tree (per sha) #2019-02-06 23:20:02 kenny It wouldn't be checking out anything new - everything is all on disk. Just changing the SHAs back and forth. #2019-02-06 23:20:19 alexmiller the git objects are mostly append-only so a fetch should only need to be new stuff, small diffs #2019-02-06 23:20:34 alexmiller working tree could be slower, but it’s all in the objects #2019-02-06 23:21:16 alexmiller I guess I would compare doing a git clone or fetch separately and see if it’s substantially different #2019-02-06 23:22:02 alexmiller there’s caching in many places here - git objects, git working tree per sha, maven deps in maven repo, and classpath in cpcache #2019-02-06 23:23:02 alexmiller so I guess I’m curious what part of that isn’t cached when it’s slow (and why) #2019-02-06 23:23:34 kenny For example, each one of these steps takes 3-5s:
Expanding compute/logger {:git/url
There are several of those. #2019-02-06 23:24:07 alexmiller is it correct that there is a newer sha? #2019-02-06 23:25:49 kenny Yes, 319 is newer than 9f9f.
Just ran time on the -Spath command. It actually took 52s for a CP refresh. #2019-02-06 23:27:20 kenny I should clarify - that is one example of the messages that takes a while. AFAICT, there are no duplicates in the log. #2019-02-06 23:55:46 alexmiller so it should be fetching the git repo, then cloning the working tree. Are there maybe big binary objects in the working tree? One idea I had with gitlibs was to have the ability to narrow the working tree to just a subset of the full dir (so you could exclude like an images directory or something was irrelevant to the Clojure source) #2019-02-06 23:56:49 alexmiller still not entirely sure that’s the slow bit from your description but maybe a testable hypothesis #2019-02-11 17:26:31 borkdude That proposal introduces the ability to have managed deps with minimal change to the deps.edn spec (don’t know about the impl) #2019-02-12 09:34:22 martinklepsch When using :local/root with a .jar will that resolve transitive dependencies of the jar by inspecting the pom.xml inside that jar?
And is it standard that pom.xml is included in the jar? #2019-02-12 09:37:31 borkdude Is that supposed to work? I’m surprised #2019-02-12 09:38:21 borkdude Yeah, it’s documented: Local jar: {:local/root "/path/to/file.jar"}. TIL #2019-02-12 09:39:58 martinklepsch Yeah, it's pretty neat. I just wasn't aware that pom.xml is always(?) part of the jar so thought that this couldn't work with transitive deps. #2019-02-12 13:46:44 alexmiller It is normally part of the jar. #2019-02-12 14:22:59 stijn what might be the reason that it takes a very long time to get a REPL up? I just upgraded my ArchLinux and before it took a few seconds, now, minutes. I believe the only change is the upgrade of the jdk 8.u192-1 -> 8.u202-1. #2019-02-12 14:24:35 stijn If I don't include the 'dev' profile, which has some machinery with tools.namespace (and a logback.xml) in it, it goes back to seconds. #2019-02-12 14:33:47 alexmiller well the main thing it’s doing is network I/O so that would be my default area of concern #2019-02-12 14:34:18 alexmiller do some stack traces with ctrl-\ while it’s working and see what it’s doing #2019-02-12 15:07:21 alexmiller 8.u201 (and 202) include new buffer overrun checks in Linux, not sure if that would affect visible perf: “Additional safeguards to protect against buffer overruns in native code have been enabled on Linux. If a buffer overrun is encountered the system will write the message “stack smashing detected” and the program will exit. Issues of this type should be reported to your vendor.” #2019-02-12 16:28:41 stijn if I switch to java 11 it's back to 'seconds' #2019-02-12 16:30:51 stijn it looks like the compilation of namespaces that are required from user is slow. when these are uncommented, it's fast on 8.u202 too #2019-02-12 17:20:30 alexmiller hmm, still not clear to me what the difference is #2019-02-12 17:21:55 bronsa could you post your user.clj maybe? #2019-02-13 07:26:53 stathissideris does the Clojure runtime have access to the aliases that were passed? Can my code access this information somehow? #2019-02-13 07:31:09 dominicm There are some system properties set. I think only clojure.libfile #2019-02-13 07:31:17 dominicm Why do you need to know? #2019-02-13 07:50:10 stathissideris I’d like to know whether its a figwheel run or not #2019-02-13 07:59:08 dominicm @stathissideris everyone is going to name their Figwheel alias differently. I'd check if the namespace is loaded/can be required, and depending on context, check if the state var is bound. #2019-02-13 07:59:56 dominicm Your code might work with some lein setups too that way #2019-02-13 08:04:59 stathissideris good point about about naming, but I need this for my specific setup, I’m not making a generic tool, so hardcoding it would be ok #2019-02-13 08:06:11 stathissideris Is is possible to combine -e and -m to inject a var maybe? #2019-02-13 08:13:52 dominicm I do this in edge using :jvm-opts #2019-02-13 08:14:14 dominicm Jvm opts compose, main opts do not. #2019-02-13 08:15:11 dominicm Using -D in jvm opts that is. #2019-02-13 08:17:08 stathissideris oh thanks, I’ll check it out #2019-02-13 13:40:22 dominicm I think there's a NPE in t.d.a:
Error building classpath. nil
java.lang.NullPointerException
at clojure.tools.deps.alpha.extensions.maven$eval638$fn__641.invoke(maven.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$canonicalize_deps$fn__1027.invoke(alpha.clj:74)
at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)
at clojure.core$reduce.invokeStatic(core.clj:6827)
at clojure.core$reduce.invoke(core.clj:6810)
at clojure.tools.deps.alpha$canonicalize_deps.invokeStatic(alpha.clj:73)
at clojure.tools.deps.alpha$canonicalize_deps.invoke(alpha.clj:71)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:170)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:236)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:218)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invokeStatic(make_classpath.clj:59)
at clojure.tools.deps.alpha.script.make_classpath$create_classpath.invoke(make_classpath.clj:52)
at clojure.tools.deps.alpha.script.make_classpath$run.invokeStatic(make_classpath.clj:70)
at clojure.tools.deps.alpha.script.make_classpath$run.invoke(make_classpath.clj:64)
at clojure.tools.deps.alpha.script.make_classpath$_main.invokeStatic(make_classpath.clj:109)
at clojure.tools.deps.alpha.script.make_classpath$_main.doInvoke(make_classpath.clj:84)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:491)
at clojure.main$main_opt.invoke(main.clj:487)
at clojure.main$main.invokeStatic(main.clj:598)
at clojure.main$main.doInvoke(main.clj:561)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:37)
this may be time sensitive as I'm using a just-deployed coordinate. #2019-02-13 13:40:41 dominicm org.webjars.bower/react {:mvn/version "15.4.2"}
org.webjars.bower/es6-promise {:mvn/version "4.0.5"}
org.webjars.bower/fetch {:mvn/version "0.9.0"}
org.webjars.bower/graphiql {:mvn/version "0.11.11"}
org.webjars.npm/graphiql-subscriptions-fetcher {:mvn/version "0.0.2"}
org.webjars.npm/subscriptions-transport-ws {:mvn/version "0.8.3"}
these are the deps I just added #2019-02-13 13:42:18 dominicm org.webjars.npm/graphiql-subscriptions-fetcher {:mvn/version "0.0.2"} this appears to be the exact problem dep #2019-02-13 13:51:28 alexmiller runtime does not have access to the aliases that were used @stathissideris #2019-02-13 13:53:12 stathissideris @alexmiller thanks, I ended up using -D to indicate the alias via properties as @dominicm suggested #2019-02-13 13:53:59 alexmiller I can repro the npe, will take a look. shouldn’t be timing-related #2019-02-13 13:54:22 alexmiller there’s a version range in the transitive deps, wouldn’t surprise me if that’s involved #2019-02-13 13:55:56 dominicm that seems more likely, yeah. I was worried that we'd caught maven in an inbetween state and it was causing an NPE. #2019-02-13 13:56:11 alexmiller I think it’s finding no matches in the version range #2019-02-13 13:56:40 alexmiller so code could definitely be tightened up around that to not npe but it will just not find the dep then #2019-02-13 13:57:34 dominicm yeah. makes sense. webjars is a bit weird like this I guess 🙂 #2019-02-13 13:57:41 dominicm I can add explicit graphiql dep here. #2019-02-13 13:58:29 alexmiller org.webjars.npm/graphiql-subscriptions-fetcher {:mvn/version "0.0.2"} depends on org.webjars.npm/graphql {:mvn/version "[0.9.1,0.10)" but I don’t see anything older than 0.10.1 in maven central #2019-02-13 13:58:59 alexmiller I’ll fix the npe #2019-02-13 13:59:05 dominicm webjars is a bit weird. You basically are deploying npm things. So if nobody has deployed graphiql 0.10, then it won't be there. #2019-02-13 13:59:29 dominicm adding an explicit graphiql dep hasn't helped though, annoyingly. So I guess I will deploy 0.10 via http://webjars.org #2019-02-13 13:59:39 alexmiller does npm actually have jars in it? #2019-02-13 14:00:00 dominicm webjars takes stuff from npm, and puts it in a jar #2019-02-13 14:00:28 alexmiller tdeps could have an extension to use webjars directly #2019-02-13 14:00:54 dominicm do you mean npm directly? I don't know how you'd use webjars directly 🙂 #2019-02-13 14:01:55 dominicm but either way, that would be a huge +1 from me #2019-02-13 14:02:35 alexmiller well, you would need to tell me more things for me to understand what’s possible, but anything that can be a source of named, versioned artifacts is a potential target #2019-02-13 14:08:13 dominicm good question, admittedly I don't fully understand how webjars works. #2019-02-13 14:08:17 alexmiller well, webjars seems to have a a cdn? whatever step you’re doing seems like some sort of mindless wrapping task #2019-02-13 14:08:21 alexmiller programs are good at those #2019-02-13 14:09:24 dominicm > All of the WebJar contents are available on the public jsDelivr CDN. Just prefix //cdn.jsdelivr.net/webjars/{groupId} in front of your static asset URLs. For instance, if using the org.webjars : jquery WebJar and your local URL to jquery.js is /webjars/jquery/2.1.0/jquery.js then the CDN URL would be: //cdn.jsdelivr.net/webjars/org.webjars/jquery/2.1.0/jquery.js
So I guess the contents of the jars are available via the CDN, rather than the jars themselves. #2019-02-13 14:09:41 dominicm the actual jars go to maven. #2019-02-13 14:09:55 dominicm then you can request a new jar to go to maven via clicking buttons on http://webjars.org - that triggers an automated process #2019-02-13 14:13:11 alexmiller “clicking buttons” :) #2019-02-13 14:14:31 dominicm is your proposal that instead of deploying to maven, a webjars-like thing would build the jars and stick them in ~/.local/cache/webjars/foo.bar.1.jar ? #2019-02-13 14:15:47 alexmiller why do you even need jars? they’re just files on the classpath right? #2019-02-13 14:16:39 alexmiller it would be helpful to do something that actually helped people down the line as they build and deploy stuff #2019-02-13 14:17:08 alexmiller I am too ignorant about cljs / npm / node / etc to not know what would be useful #2019-02-13 14:17:51 dominicm you're right. jars not needed. #2019-02-13 14:17:58 alexmiller npm is a central repository. it has stuff you want in your app. tools.deps has an extension point to plug in code to get stuff and put it on your classpath (and determine its dependencies) #2019-02-13 14:17:59 dominicm fwiw, this is pretty unrelated to cljs. #2019-02-13 14:18:09 alexmiller see how ignorant I am? :) #2019-02-13 14:18:37 dominicm I think npm is a good target for t.d.a. So is bower. #2019-02-13 14:18:48 dominicm maybe not bower, it's kinda old now. #2019-02-13 14:19:13 alexmiller yeah, anything that didn’t come out this month in js is old :) #2019-02-13 14:20:14 alexmiller I guess coming back to a problem statement - this is like js stuff that you want to serve as static resources from your app? #2019-02-13 14:24:10 alexmiller does npm really not have any kind of namespacing? #2019-02-13 14:27:44 alexmiller oh, it has scopes, I see #2019-02-13 14:28:40 alexmiller well, seems like the info necessary exists if you want to file a ticket defining what’s useful #2019-02-13 14:29:54 dominicm > I guess coming back to a problem statement - this is like js stuff that you want to serve as static resources from your app?
Exactly! #2019-02-13 14:30:05 dominicm Although, admittedly, I'd personally be quite happy with cdn integrations. #2019-02-13 14:30:51 dominicm bower predates npm afaik. It was a big step forward at the time, but it's not really used. npm got more traction and was dominant by the time a majority of companies turned their attention to js. #2019-02-13 14:31:16 dominicm bower doesn't recommend you use bower so ¯\(ツ)/¯ #2019-02-13 14:31:42 dominicm I'll write down that I want to create this, and have a note for myself to think a little more about what I want 🙂 #2019-02-13 14:32:51 dominicm @alexmiller while I have you, is there a workaround for this graphiql issue? bintray appears to be having issues, and I can't get an older graphiql to deploy. :exclusions hasn't worked, nor has an explicit graphiql dependency on 0.11.11. #2019-02-13 14:35:37 alexmiller not sure I have a good option for you #2019-02-13 14:36:34 alexmiller if you made webjars deploy version 0.9.1 to maven central that would let it proceed #2019-02-13 14:37:10 dominicm Unfortunately, that's failing 🙂 I thought you might say that though. It's okay #2019-02-13 14:40:09 alexmiller kind of an interesting failure mode. would be better if it could defer long enough to be overridden #2019-02-13 14:40:56 alexmiller in this particular case, the only reason it’s resolving early is due to the version range I think, trying to narrow that down to single version #2019-02-13 16:43:23 dominicm #2019-02-13 16:43:57 dominicm graphiql vs graphql #2019-02-13 17:17:20 dominicm You hit an NPE whenever you have a transitive dependency which doesn't exist. I'm seeing it with a version pinned to [1.0.3] (maybe that is a range though?) #2019-02-13 17:19:35 dominicm Actually, no. I'm confused. #2019-02-13 17:20:17 dominicm I've rm -rf .cpcache, and used -Sforce, just in case. #2019-02-13 17:20:36 dominicm ❯ clj -Sdeps '{:deps {org.webjars.npm/graphql {:mvn/version "0.9.1"}} :aliases {:v {:verbose true}}}' -A:v -Sforce
Initial deps to expand:
{org.clojure/clojure #:mvn{:version "1.10.0"},
org.webjars.npm/graphql #:mvn{:version "0.9.1"}}
Expanding org.clojure/clojure #:mvn{:version 1.10.0}
=> include, pin top dep
Expanding org.webjars.npm/graphql #:mvn{:version 0.9.1}
Error building classpath. nil
java.lang.NullPointerException
at clojure.tools.deps.alpha.extensions.maven$eval638$fn__641.invoke(maven.clj:41)
#2019-02-13 17:35:22 dominicm [ERROR] Failed to execute goal on project my-app: Could not resolve dependencies for project : Failed to collect dependencies at org.webjars.npm:graphql:jar:0.9.1 -> org.webjars.npm:iterall:jar:[1.0.3]: No versions available for org.webjars.npm:iterall:jar:[1.0.3] within specified range -> [Help 1] maven is struggling too #2019-02-13 17:36:47 dominicm Oh. Interesting. It is fine with it as a top-level dependency, but not as a transitive dep. #2019-02-13 18:17:32 alexmiller [1.0.3] is a version range. from Maven’s perspective just 1.0.3 by itself is a specific version #2019-02-13 18:20:55 alexmiller I don’t think there’s any difference with top vs transitive - you probably just have it in your local Maven repo at ~/.m2/repository now #2019-02-13 18:22:50 alexmiller it works for me as a transitive dep, and as a top-level dep with specific version, and as a top-level dep with version range [1.0.3] #2019-02-13 18:22:59 alexmiller (clearing all caches before each one) #2019-02-13 18:24:53 alexmiller $ rm -rf ~/.m2/repository/org/webjars
$ clj -Sdeps '{:deps {org.webjars.npm/graphql {:mvn/version "0.9.1"}}}' -Sforce
Downloading: org/webjars/npm/graphql/0.9.1/graphql-0.9.1.pom from
Downloading: org/webjars/npm/iterall/maven-metadata.xml from
Downloading: org/webjars/npm/iterall/1.0.3/iterall-1.0.3.pom from
Downloading: org/webjars/npm/graphql/0.9.1/graphql-0.9.1.jar from
Downloading: org/webjars/npm/iterall/1.0.3/iterall-1.0.3.jar from
Clojure 1.10.0
user=> #2019-02-13 18:27:21 alexmiller when you deploy to Maven central it can take 10 minutes to an hour to show up for download, so maybe it just wasn’t there yet #2019-02-13 18:37:46 dominicm I'm a bit suspicious that perhaps version ranges are searched somewhere different than specific versions. #2019-02-13 18:41:11 alexmiller they aren’t #2019-02-13 18:41:24 alexmiller but there is an extra piece of code to explicitly resolve them in tdeps #2019-02-13 18:42:09 alexmiller if you haven’t, you should try selectively deleting part of your ~/.m2/repository/org/webjars/… #2019-02-13 18:42:32 alexmiller there are metadata files about versions that are cached there and only updated once every 24 hrs #2019-02-13 18:42:55 alexmiller (this is a setting we don’t currently surface - if you’re using mvn, you can pass -U to force an update) #2019-02-13 18:43:52 dominicm Is it possible that metadata would store [1.0.3] separately from 1.0.3? #2019-02-13 19:01:19 alexmiller it just stores known versions and then uses that to resolve #2019-02-13 19:01:41 alexmiller but if you had already filled that cache for the day and the repository said that version didn’t exist, it’s going to keep telling you that #2019-02-13 19:01:51 alexmiller it will not go re-check the repository #2019-02-13 19:02:37 alexmiller this file in particular: ~/.m2/repository/org/webjars/npm/iterall/maven-metadata-central.xml #2019-02-13 19:03:46 alexmiller last check timestamps are in ~/.m2/repository/org/webjars/npm/iterall/resolver-status.properties #2019-02-13 19:59:31 stijn @alexmiller on the slow REPL startup of the latest java 8 (u202) on Linux, looking at the stack traces, it spends a lot of time compiling some java-time things (https://github.com/dm3/clojure.java-time). One CPU goes to 100% during this and after that stack trace is gone, the REPL starts. #2019-02-13 20:08:23 hiredman java-time does lot of meta programming stuff, finding types, doing a graph traversal to find paths between types #2019-02-13 20:08:55 hiredman so I could imagine types changing between jdk versions #2019-02-13 20:13:19 dominicm blugh, after all this work, one of my transitive deps doesn't work because it has a custom gql license which webjars won't automatically repackage because it doesn't recognize. If we were a direct npm client this would be avoided. Very tempting proposition really. #2019-02-13 20:13:46 dominicm @alexmiller if I wrote an npm client with an EPL license in clojure, would it be candidate for use in t.d.a? Or would you prefer it was under the clojure/core umbrella? #2019-02-13 20:42:38 alexmiller could be an external lib (we use others already). I think there are some out there already. but maybe the amount of access needed by tda is so small to not even be worth that, not sure. #2019-02-13 20:43:40 alexmiller I guess you probably want a cache ala tools.gitlibs though and that’s something we’d want outside tda #2019-02-13 20:46:30 dominicm I haven't really looked at the npm client details yet. Might be worth seeing if we can share the yarn cache. Plus I'm a bit uncertain about how to handle http. #2019-02-13 20:47:07 alexmiller well I think you have the same options as gitlibs - shell out to npm or make a direct implementation #2019-02-13 20:49:07 alexmiller pros and cons. might be worth gathering some more information about the things you need to implement:
1) given a name/version, determine that lib’s deps (don’t need transitive, just shallow)
2) given a name/version, download the bits somewhere #2019-02-13 20:49:12 dominicm Npm is far more amenable I think. Yarn is a reimplementation of the npm client. Plus, npm doesn't have a global cache, and because of the hierarchical nature, you probably want more control. #2019-02-13 20:49:49 dominicm I suppose you have to figure out auth, which is the recurring issue. #2019-02-13 20:53:01 alexmiller well public stuff is fine I assume #2019-02-13 20:53:22 alexmiller for creds, we already have a means to do this via the maven settings.xml on a per-repo basis #2019-02-13 20:53:43 alexmiller admittedly kind of weird to put npm auth there #2019-02-13 20:19:17 stijn @hiredman, the weird thing is that it's only a minor version upgrade (from 8.u192 to 8.u202). With 192 the repl starts in seconds, with 202 it takes minutes #2019-02-13 20:21:04 ghadi I know it's aside from your problem @stijn but I wouldn't use that library, to be perfectly honest. #2019-02-13 20:21:27 ghadi java.time is perfectly approachable #2019-02-13 20:21:27 ghadi java.time is perfectly approachable #2019-02-13 20:29:44 seancorfield That's a matter of opinion @U050ECB92 🙂 I find clojure.java-time really does produce cleaner-looking code in a lot of situations. #2019-02-13 21:17:40 ghadi clojure.java-time seems to have a steep cost in classloading and metaprogramming #2019-02-13 21:19:43 seancorfield It's a one-off cost, the first time you use certain constructors. It's fast after that. #2019-02-13 21:20:06 seancorfield The readme even shows how you can "warm up" that stuff. #2019-02-13 21:20:49 seancorfield I was a bit surprised that it caused such a "blip" on 202(?) since it's fine on 192 (and on subsequent builds) according to the OP. #2019-02-14 08:47:30 stijn I will investigate some more when I have the time. For now, went back to 192. #2019-02-13 20:21:39 ghadi the actual JDK package java.time #2019-02-13 20:24:08 hiredman maybe check to see if you have org.threeten.extra.Temporals and or org.joda.time.DateTime on your classpath, and if calling (Class/forName ...) with those class names changes behavior on the different jdk versions #2019-02-14 08:50:08 stijn yes org.joda.time.DateTime is on the classpath. will check later what the impact is #2019-02-13 20:46:36 alexmiller just a couple small enhancements, no biggie #2019-02-13 20:46:55 dominicm Woah, very quick, thanks! #2019-02-13 20:49:55 alexmiller hadn’t released in a while #2019-02-13 21:25:16 seancorfield That Groups post hasn't gone out via email again, as far as I can see @alexmiller #2019-02-13 21:27:27 seancorfield (I just replied to it on the web which should cause it to be seen via email 🙂 ) #2019-02-13 22:12:52 alexmiller I sent that one from a different account #2019-02-14 19:40:15 avi looks like the brew PR has been merged — I just ran brew upgrade and my Clojure install got upgraded to 1.10.0.414 #2019-02-14 19:52:17 alexmiller yep, it's out there now #2019-02-14 20:00:57 avi 🎉 thanks Alex! aw_yeah #2019-02-20 14:53:01 zetafish Does anyone know how to change the local maven repo folder when running clj so that jars are downloaded and searched for in a different folder than ~/.m2 #2019-02-20 15:08:26 alexmiller in deps.edn, set :mvn/local-repo "/foo/bar" at top level #2019-02-20 16:23:29 zetafish Thanks @alexmiller that works. Sadly enough it seems not possible to put this option in the :aliases. My problem is that I want the maven repo to be local to my source code when I run unit tests on my CI server. While when developing locally I want to just use the default ~/.m2. Are there plans to allow the key also in the :aliases? Or maybe I should revert to tricks like rewriting the deps.edn file on the CI server prior to running tests etc (ugh) #2019-02-20 16:25:03 alexmiller you can use -Sdeps '{:mvn/local-repo "whatever"}' just on CI #2019-02-20 16:25:13 zetafish Cool! I am just a beginner with tools-deps 🙂 #2019-02-20 16:59:04 dominicm My friend got this error when using brew on his mac, I don't use mac, so don't really understand why gcc is related to clojure at all?
Error: An exception occurred within a child process:
CompilerSelectionError: clojure cannot be built with any available compilers.
Install GNU's GCC
#2019-02-20 17:00:11 alexmiller seems bad #2019-02-20 17:00:19 alexmiller brew doctor is the generic "why is brew broken?" #2019-02-20 17:00:49 alexmiller brew install clojure should not require anything to be compiled #2019-02-20 17:01:30 alexmiller brew update updates brew itself #2019-02-20 17:02:01 dominicm Does brew generally work without xtools? #2019-02-20 17:07:14 dominicm installing xtools solved it, so I guess that's a dependency? /shrug #2019-02-20 17:11:16 alexmiller I never understood how xtools is exactly involved, I guess to allow brew to compile stuff #2019-02-20 17:13:11 dominicm yeah, but, unless it compiles rlwrap from source, I have no idea what it would be building. #2019-02-20 17:28:27 alexmiller Me neither #2019-02-21 16:39:25 stathissideris does this look wrong somehow? It’s an alias in my deps.edn
{:main-opts ["-e" "(require 'dev) (dev/seed-db)"]}
and it throws a
Exception in thread "main" clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: EOF while reading, starting at line 1 #2019-02-21 16:59:53 alexmiller yeah, the inner string is not going to end up getting quoted right #2019-02-21 17:02:29 stathissideris so it just needs more quoting? {:main-opts ["-e" "\"(require 'dev) (dev/seed-db)\""]} #2019-02-21 17:03:20 stathissideris that doesn’t work either #2019-02-21 17:03:22 alexmiller no, that probably won't work either due to how this gets propagated. this is a known issue #2019-02-21 17:03:38 alexmiller you can try the "corfield comma" (comma instead of spaces) #2019-02-21 17:04:01 alexmiller {:main-opts ["-e" "(require,'dev)(dev/seed-db)"]} #2019-02-21 17:04:54 stathissideris yes, that does work, thank you very much — I don’t think I would have found it myself! #2019-02-21 17:42:16 stathissideris I’ve been slowly figuring out many of these myself, so many thanks for this treasure trove! #2019-02-22 08:13:14 Chris This is great! For your rebl-11 alias, does that remove the need to install OpenJFX? #2019-02-22 09:17:59 Chris Well, it does! I just tried 🙂 #2019-02-24 02:55:07 maxp How to make uberjar like depstar does but with aot and manifest, optionally excluding sources? #2019-02-25 09:05:20 Chris Wow Badigeon looks great, nice. #2019-02-25 14:01:36 maxp Cambada looks great but there was some critical issue - it failed to make jar with cheshire 5.8.1 that comes with new Jackson, with cheshire 5.8.0 it wokrs well. #2019-02-25 14:05:25 maxp Ok, I'll try badigeon later.
At that time I just do (compile ') and depstar classes into jar 🙂 #2019-02-26 09:19:15 maxp It works well when I use cheshire 5.8.0 in project dependencies but fails with cheshire 5.8.1!
The difference is in version of Jackson used by Cheshire. #2019-02-25 12:08:16 souenzzo few days ago i saw it on twitter a project that use's tools-deps + lein as library to generate uberjar but I lost the link 😞 #2019-02-25 12:08:34 dominicm Not lein-tools-deps do you mean? #2019-02-25 12:08:46 dominicm or pack.alpha / depstar maybe? #2019-02-27 20:20:06 pbrown Hey there! I have a small question about intended use/behavior:
Should libraries declare [:paths ["src"]] in their deps.edn when they use the default?
If my deps.edn includes a top-level [:paths ["bogus/sources"]] entry, and I bring a lib in via :git/url that does not declare :paths in its own deps.edn, its :paths-derived classpath entry becomes ~/.gitlibs/libs/$GROUP_ID/$ARTIFACT_ID/$SHA/bogus/sources, which doesn't exist.
For example:
clj -Srepro -Sdeps '{:paths ["bogus/sources"] :deps {com.bhauman/figwheel-core {:git/url "" :sha "840f05547935f65a28c5a3c640a9473961da5636"}}}' -Spath | tr : '\n' | grep figwheel
# => /home/phil/.gitlibs/libs/com.bhauman/figwheel-core/840f05547935f65a28c5a3c640a9473961da5636/bogus/sources
#2019-02-27 20:20:32 pbrown Or is this something tools.deps could/should do (would it be safe to assume we all use the same /usr/local/lib/clojure/deps.edn)? #2019-02-27 20:24:18 alexmiller This a known bug #2019-02-27 20:24:57 alexmiller Where the current project paths are used with dependency projects on the file system #2019-02-27 20:28:33 pbrown ah ty, missed it the first time through. It has the same repro! #2019-03-02 02:47:28 nbenketaf Hi there, can I mix clojure and java files in the same tools deps project ?
Like :path ["src" "java-src"] where src contains clojure source files and java-src java source files, and then import the classes from java source files directly in a clj source file ? #2019-03-02 02:51:18 hiredman no #2019-03-02 02:59:32 nbenketaf ok thanks, are there some external tools/libs that can achieve that or is it planned in the tools deps roadmap ? #2019-03-02 03:01:24 hiredman as far as I know it is positively not on the road map, java requires compilation before you can use it, a build step, and tools.deps is positioned as "not a build tool" #2019-03-02 03:02:12 hiredman I would be kind shocked if there isn't a tool someone wrote for it, but I don't know of one off hand #2019-03-02 03:08:37 nbenketaf Maybe I can handle that with the “full-stack-clj-example” above, integrating leiningen as a lib, thanks anyway 🙂 #2019-03-02 03:52:48 seancorfield @nbenketaf Or you could, you know, run javac on your Java code, and then run clj... #2019-03-02 04:01:23 seancorfield Another option is this: {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "7c0f191d121f3c38392f3728332c512d294e29364d32"}, :content ("[email protected]")} #2019-03-02 05:00:23 nbenketaf @seancorfield thank you for the inputs, here is the larger problem I am trying to solve : I have a forked project A initially managed with lein which includes some java files. I tried to convert this project to a tools deps one, basically I added a deps.edn file. Then I have another project B that declares a deps to project A with a git/url coordinate, and it’s in project B where I need the compiled java code. I guess I should package my forked project A, and use the mvn coordinate instead #2019-03-02 05:09:35 seancorfield Yeah, if a dependent project has code that needs to be compiled, either you need to package it as an "artifact", or you'll need to bring it down as source and compile stuff yourself. #2019-03-02 05:10:14 seancorfield Java source projects pretty much require compiled artifacts for use. Clojure is trying to get away from that by allowing source projects to be used as-is. #2019-03-02 10:18:17 nbenketaf Ok i see, thank you for taking the time answering 🙂 #2019-03-04 21:00:48 rschmukler Is anyone using https://github.com/clojure/tools.deps.alpha/tree/add-lib effectively? I'm getting a NullPointerException on clojure.tools.deps.alpha.util.maven/make-session (maven.clj:141) - seemingly since around the time the zero-day vulnerabilities in the JVM got fixed (but I might be conflating things). No worries if the answer is "this branch isn't supported, don't use it" #2019-03-04 21:01:20 alexmiller could you be more specific about what you're doing? #2019-03-04 21:01:45 rschmukler Calling add-lib causes that to be thrown #2019-03-04 21:02:31 rschmukler Hmm actually though, not all the time #2019-03-04 21:02:54 alexmiller so, that's a branch, what commit are you using, what is the full call, etc. give me a repro. #2019-03-04 21:03:16 rschmukler calling it with clojure.java-time {:mvn/version 0.3.2} seems to be the issue. #2019-03-04 21:03:29 rschmukler @alexmiller will do - give me a moment to try and create a minimal repro repo #2019-03-04 21:04:08 alexmiller you don't need a repo - you can put it all in an clj -Sdeps call to start, then whatever expression you're doing #2019-03-04 21:16:40 alexmiller $ clj -Sdeps "{:deps
> {org.clojure/tools.deps.alpha
> {:git/url \""
> :sha \"e160f184f051f120014244679831a9bccb37c9de\"}}}"
Checking out: at e160f184f051f120014244679831a9bccb37c9de
Clojure 1.10.0
user=> (use 'clojure.tools.deps.alpha.repl)
nil
user=> (add-lib 'clojure.java-time {:mvn/version "0.3.2"})
true
user=> (require 'java-time)
Syntax error (IllegalAccessError) compiling at (temporal.clj:324:1).
class clojure.lang.PersistentUnrolledVector$Card1 tried to access field clojure.lang.APersistentVector._hash (clojure.lang.PersistentUnrolledVector$Card1 is in unnamed module of loader clojure.lang.DynamicClassLoader @7a0e1b5e; clojure.lang.APersistentVector is in unnamed module of loader 'app') #2019-03-04 21:17:43 rschmukler @alexmiller I'm not even getting that far, although I can't seem to reproduce the issue in a new repo so now I'm trying to recreate more and more of the state in my user.clj as to what's happening before I invoke add-lib. #2019-03-04 21:18:19 alexmiller I get that defaulting to Clojure 1.10. Not sure what clojure.java-time or the dependency clj-tuple assume with clj or java version #2019-03-04 21:19:14 rschmukler Interesting. I've got clojure.java-time working w/ a 1.10 repo, although I added it via deps.edn, not dynamically via add-lib - so I haven't tested it. #2019-03-04 21:21:12 alexmiller clj-tuple seems to be compiled Java classes, certainly possible that doesn't play nice with add-lib #2019-03-04 21:21:39 alexmiller that error doesn't match what you described though re NPE #2019-03-04 21:21:45 rschmukler I'm seeing it w/ other libs as well - but I'm hitting the error while invoking add-lib, not after (and trying to require it) #2019-03-04 21:22:08 alexmiller what sha are you using for tools.deps? #2019-03-04 21:22:21 rschmukler e160f184f051f120014244679831a9bccb37c9de #2019-03-04 21:23:30 rschmukler Also it seems to raise the error w/ about 50% of the deps. #2019-03-04 21:23:55 rschmukler ie. I wrote a wrapper that calls add-lib on my project for everything in deps.edn #2019-03-04 21:23:59 rschmukler (defn sync-libs
"Dynamically refreshes all libs in `deps.edn` and installs any missing"
([] (sync-libs nil))
([?alias]
(let [edn (-> "deps.edn" slurp read-string)
libs (if ?alias
(get-in edn [:aliases ?alias :extra-deps])
(get edn :deps))]
(doseq [[lib-sym lib-spec] libs]
(try
(add-lib lib-sym lib-spec)
(catch NullPointerException e
(println "Error raised for:" lib-sym lib-spec)))))))
#2019-03-04 21:24:46 rschmukler But again, this doesn't appear to be an issue in a different project. Still trying to build back up to it happening. #2019-03-04 21:28:17 alexmiller add-lib doesn't re-add any deps it already has btw #2019-03-04 21:28:45 rschmukler That's fine, I was presuming it to be a no-op #2019-03-04 21:29:17 rschmukler (ie. my workflow is I have a file watcher watching deps.edn and it syncs the libs on change. I'm not doing any diffing) #2019-03-04 21:29:43 alexmiller the error seems to imply you can't make a maven system, but not sure why you wouldn't get an error at that point #2019-03-04 21:30:04 alexmiller (use 'clojure.tools.deps.alpha.util.maven)
(make-system)
#2019-03-04 21:31:38 rschmukler That worked (ie. no exception) #2019-03-04 21:33:31 alexmiller well, not really sure. you're doing this under nrepl and it's possible the classloader hierarchy there is having some impact, don't really know. #2019-03-04 21:33:45 alexmiller I haven't seen this reported before (although obviously not many people using this) #2019-03-04 21:35:00 rschmukler I'm actually seeing it even if I don't use nrepl (ie. if I just use a plain ol' clojure repl) #2019-03-04 21:35:19 alexmiller well, that's useful in ruling things out #2019-03-04 21:35:41 rschmukler But yeah, I appreciate that I'm beyond the bleeding edge w/ this. I'll see if I can reproduce it... #2019-03-04 21:35:47 alexmiller is there any chance you have this happening in multiple threads a the same time? #2019-03-04 21:36:23 rschmukler I don't think so. I thought it might be some sort of race condition but it seems to be consistent in the dependencies that it fails on #2019-03-04 21:36:54 rschmukler (and since it's just a clojure repl, I can't imagine anything spawning threads) #2019-03-04 21:38:41 alexmiller you could try (dotimes [_ 20] (assert (not (nil? (make-session (make-system) default-local-repo))))) #2019-03-04 21:39:01 alexmiller see if you get any nils #2019-03-04 21:40:46 rschmukler Alright, will try momentarily #2019-03-04 21:43:14 alexmiller I guess would also be good to know which java version you are on #2019-03-04 21:45:31 rschmukler I found the library that causes it #2019-03-04 21:46:10 rschmukler So, if you have an existing deps.edn w/ metasoarous/oz {:mvn/version "1.5.6"} and then try and invoke add-lib the errors happen #2019-03-04 21:46:20 rschmukler confirming in my other repo #2019-03-04 21:48:20 rschmukler Note that the sync-libs is commented out. Digging as to why that lib causes it. #2019-03-04 21:50:50 rschmukler λ java --version
openjdk 11.0.2 2018-10-16
OpenJDK Runtime Environment (build 11.0.2+7)
OpenJDK 64-Bit Server VM (build 11.0.2+7, mixed mode)
#2019-03-04 21:53:48 rschmukler Yep, this looks like it #2019-03-04 21:54:15 rschmukler > Another problem might happen if you mix lazy-require with some other code that does dynamic namespace loading. lazy-require itself makes sure to serialize the loading of namespaces so that nothing breaks because of concurrent loading (e.g., see clojure-emacs/cider#2092). However, if other code also loads namespaces in the background, something might happen. If you ever experience such issues, please file a ticket. #2019-03-04 21:55:49 rschmukler Although, it's not like I'm requiring that lib, or even using it. ie. simply having it in the project (oz, not confirmed that it's this library specifically) causes the issue. #2019-03-04 21:56:03 alexmiller well, certainly sounds likely #2019-03-04 21:58:16 alexmiller oz is using in it's notebook namespaces #2019-03-04 21:59:15 rschmukler Heh, we are doing exactly the same thing 😛 #2019-03-04 21:59:22 alexmiller actually that loads it but doesn't use it #2019-03-04 21:59:29 rschmukler Although, I don't think it's actually using it. If you look at the lib itself, it's pure #2019-03-04 21:59:58 rschmukler ie. it's not like it's doing anything to the run-time or anything #2019-03-05 22:21:36 aaelony I happen to need to use an old (but productionized) box that I’m not allowed to update java on. It’s running java 6. Both lein and clj complain about Exception in thread "main" java.lang.UnsupportedClassVersionError: clojure/main : Unsupported major.minor version 52.0
#2019-03-05 22:21:44 aaelony running on linux: #66~precise1-Ubuntu #2019-03-05 22:23:40 hiredman java 6 isn't support on recent clojure versions if I recall #2019-03-05 22:23:46 hiredman I think 8 is the new minimum #2019-03-05 22:25:10 hiredman and the change log for 1.10 says java 8 or above #2019-03-05 22:25:34 aaelony that’s probably what I’m up against. Thanks. #2019-03-05 22:26:02 aaelony maybe there’s an old version of lein I can use… #2019-03-05 22:28:26 aaelony update… found a work-around unrelated to clojure… thanks #2019-03-05 22:32:02 alexmiller yes, clj and clojure 1.10 both require Java 1.8+ #2019-03-05 22:32:35 alexmiller I assume "productionized" means "completely insecure" :) #2019-03-05 23:37:47 aaelony lol. It means I didn’t have the option of considering any change there… haha #2019-03-05 23:43:28 alexmiller Yeah, I get it #2019-03-26 22:52:28 borkdude I’m getting a stale classpath for an alias with only one local/root dependency. I added a dependency to the local/root, but now I removed it. Yet clj -A:clj-kondo -Spath still reports it being there. #2019-03-26 22:52:43 borkdude How/where can I clean this? #2019-03-26 22:53:29 borkdude -Sforce did the trick #2019-03-26 22:56:19 alexmiller Yeah, currently dep changes to local deps do not cause a recalc - need a force for that #2019-03-26 22:56:44 alexmiller I have a solution for this but it’s a bit involved #2019-03-27 20:19:15 borkdude is there an install script for MacOS like on linux? using brew on CI takes a bit long #2019-03-27 20:36:30 borkdude caching brew probably also works #2019-03-27 20:37:16 hiredman are you saying your ci server is running osx? #2019-03-27 20:37:43 borkdude yeah #2019-03-27 20:37:48 hiredman wild #2019-03-27 20:37:55 borkdude CircleCI 🙂 #2019-03-27 20:38:21 borkdude it’s pretty cool. I can build a binary for linux and MacOS that way #2019-03-27 21:04:58 alexmiller You can use the Linux script on mac I believe #2019-03-29 14:20:01 sundbp I'm trying to use a git/url dependency behind a proxy (for an https repo). Any idea how I can set the proxy info that jgit uses under the hood? #2019-03-29 14:26:51 alexmiller I think that is an open issue #2019-03-29 14:27:20 alexmiller haven't had a chance to look at it yet #2019-04-01 10:40:38 mpenet a comment says it's fixed but I couldn't confirm it, it still seems broken #2019-04-01 11:59:58 mpenet my bad it does work now #2019-04-01 19:41:16 sundbp @alexmiller I'd be willing to do some lifting on tdeps-124 (git proxies) if it is of help and we can agree a direction of where to take proxy info from. It is quite a blocker for my adoption of tools.deps so it is high prio for me. #2019-04-01 20:23:51 alexmiller go for it! I think the two main options are to leverage the existing maven info (you can look at how that's accessed in the maven extension) or to look at leveraging whatever is native for git. this probably ends up flowing down into tools.gitlibs too. #2019-04-01 20:37:57 sundbp I'm torn between using the git config as more logical, and the cost of 2 config actions required for the user. It is probably still my slight preference. #2019-04-01 21:05:10 alexmiller make a list of the tradeoffs #2019-04-01 21:05:17 alexmiller in the ticket #2019-04-02 19:38:33 sundbp @alexmiller i’m confusing myself. signed up for a JIRA account and was going to comment on that ticket but don’t seem to see any Reply type button anywhere. Is it permissioned? #2019-04-02 19:43:26 alexmiller yeah, gimme a sec #2019-04-02 19:44:16 alexmiller should work now #2019-04-02 20:36:30 sundbp thanks #2019-04-02 21:51:46 sundbp put down the 3 ways I’ve been thinking about in that ticket. #2019-04-05 13:09:14 Laverne Schrock Opinion:
Of the options presented on the ticket, all except for the "Use maven proxy" seem reasonable. A single source for the proxy config is nice, but Maven settings should only be for fetching things via Maven. If we want centralization I'd vote for the "environment variable" or the "JVM flags" options (though I'm unclear on what the latter would end up looking like). #2019-04-02 22:06:20 sundbp @alexmiller another thing I’ve been running into is the monorepo story. I’ve so far adopted the convention to “hijack” CLJ_CONFIG via wrapper tools in my monorepo. E.g. I make it impossible for the user to control CLJ_CONFIG, and use it as an escape hatch to merge in one more deps.edn into the mix. This is obviously not ideal, and leads to discussions like this one in terms of tooling etc: https://github.com/juxt/pack.alpha/pull/34. Similarly to -Sdeps, is there any milage in a similar flag taking an actual additional deps.edn file? e.g. -Sdeps-file which can be given 0 or more times. That’d make at least the parts of the monorepo cases I’ve considered more manageable. Has it already been considered? If not any initial thoughts? #2019-04-02 22:07:52 alexmiller interested in doing more, haven't had a chance to think seriously about it #2019-04-03 03:26:00 lilactown Any examples using deps.edn + packr? #2019-04-03 13:01:03 sundbp I seem to run into some classloader issue when compiling gen-class using clj. #2019-04-03 13:01:29 sundbp I'm trying to use compile with a gen-class. I get a class not found exception when the constructor is defined taking a class from a lib I'm using. I can import that same class, and changing the type to eg String makes compile work. Any caveat about types for constructor args and class path etc? #2019-04-03 13:01:43 sundbp The same setup with lein works OK. #2019-04-03 13:03:38 sundbp It is like the classloader is out of whack. If I start a rebel repl I can import the class etc, and I can compile from the repl. But when passed via the -e flag the same expression gives class not found exception. #2019-04-03 13:10:03 sundbp If I wrap it into something simple that does the same thing via a -m main method it is OK. But the -e version with same code fails. #2019-04-05 21:11:28 rafael I'm seeing an Error building classpath message when running clj. I'm running with -Sverbose but it doesn't help to identify the problematic dependency. #2019-04-05 21:13:26 rafael $ clj -Sverbose -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.9.0"} midje {:mvn/version "1.9.7"}}}'
version = 1.9.0.329
install_dir = /nix/store/i2zsj4qknknpfrvfhpx0z69c44b0c12j-clojure-1.9.0.329-prefix
config_dir = /home/rafael/.clojure
config_paths = /nix/store/i2zsj4qknknpfrvfhpx0z69c44b0c12j-clojure-1.9.0.329-prefix/deps.edn /home/rafael/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/2657530762.cp
Refreshing classpath
Error building classpath. In the dependency graph, library org.clojure/clojure specifies version range "[1.2.1],[1.3.0]" - please specify a version in your configuration.
#2019-04-05 21:13:48 rafael Should I open an issue? I'm having a hard time finding out how to proceed debugging the problem. #2019-04-05 21:14:43 alexmiller Update your clj version #2019-04-05 21:15:24 alexmiller Some older versions like this one didn’t handle version ranges #2019-04-05 21:15:32 alexmiller Newer ones do #2019-04-05 21:17:13 rafael Newbie question: is there a way to test a new clj version without updating the binary installed in the system? #2019-04-05 21:27:11 lilactown I think it's literally a shell script, you could try downloading it and executing it in a local dir? #2019-04-05 21:28:41 alexmiller It’s both a script and an uberjar #2019-04-05 21:29:37 alexmiller They need to be matched up #2019-04-05 21:31:07 alexmiller There isn’t an easy way to do what you’re asking but you could always rollback to the version you’re on - it’s just the same url with the version you want #2019-04-05 21:31:55 rafael Thanks! #2019-04-08 13:15:21 stathissideris deps has made such a big difference to my Clojure workflow! I’m eternally grateful for Leinignen for taking us this far, but tools-deps has made things even better #2019-04-08 13:21:34 alexmiller thx! #2019-04-08 14:31:17 stathissideris I’ll list some specific benefits that I’ve seen:
1. Less mono-repo oriented development due to effortless git deps (this was huge for me).
2. I’m more likely to fork open source libraries (and attempt to fix them) also because of git deps.
3. Casually adding custom automated project-related “tasks” (DB migrations, linting, testing etc) because of the combination of aliases and Clojure “scripts” (where with lein I would need a plugin)
4. Stopped using project generation tooling, I now make a deps.edn, an src folder and I suddenly have a new Clojure project! #2019-04-08 14:39:48 kirill.salykin > 3.
cant you do same with lein alias? #2019-04-08 14:49:09 souenzzo yeah, but lein culture says to develop a plugin. #2019-04-08 15:05:47 stathissideris to the point where I was not aware of lein alias 😄 #2019-04-08 15:12:51 stathissideris oh I did know it, but it requires a main method #2019-04-09 16:15:15 borkdude how can I add compiled .class files to the classpath with tools.deps? #2019-04-09 16:17:20 dominicm Put them in a folder, add it to your paths. #2019-04-09 16:17:42 borkdude so I put it in target/classes and then :paths ["src" "target/classes"]? #2019-04-09 16:18:32 alexmiller yep #2019-04-09 16:18:37 borkdude I tried this from the command line: -J-cp target/classes but that didn’t work #2019-04-09 16:18:51 alexmiller no, that won't work as clj is building the classpath #2019-04-09 16:19:00 alexmiller probably should actually make that an error #2019-04-09 16:19:12 borkdude and if target/classes doesn’t exist, it won’t bail out? #2019-04-09 16:19:14 alexmiller that's the one java property you can't set :) #2019-04-09 16:19:23 alexmiller missing dirs are fine #2019-04-09 16:19:26 borkdude cool #2019-04-09 16:19:28 alexmiller java doesn't care #2019-04-09 16:19:51 alexmiller (but it also won't pick up stuff that gets added later, I don't think) #2019-04-09 16:19:57 borkdude that’s fine #2019-04-09 16:20:17 borkdude I only need to compile a class if I’m using GraalVM compilation, for normal JVM usage it’s unnecessary #2019-04-09 16:20:28 alexmiller yeah, should be fine #2019-04-09 16:20:30 borkdude so this dir is going to be in my gitignore #2019-04-09 16:20:45 alexmiller makes sense #2019-04-09 17:16:26 rschmukler Is it possible to set environment variables via the deps.edn file, similar to the :env section of a project.clj? If not, what's the recommended way to handle that when using deps.edn? #2019-04-09 17:19:55 hiredman :env as a way to set env variables is something some lein plugin provides #2019-04-09 17:20:57 hiredman I don't think that plugin actually sets environment variables, it provides its own kind of shadow system that you get access to as long as you use that plugin's functions for reading env variables #2019-04-09 17:21:39 rschmukler Ah! I misunderstood. There was a github issue that made it seem like it did something better than that -namely they were overwriting LD_PRELOAD and it was working it seemed #2019-04-09 17:22:33 rschmukler (I'm hooking into some C FFI with an old relic of a library) #2019-04-09 17:23:24 hiredman it looks like environ dumps the environment stuff to .lein-env, which is maybe a standard thing? I am not sure #2019-04-09 17:23:47 hiredman but like, if you have environment variables, why not just set them? #2019-04-09 17:24:08 hiredman LD_PRELOAD=... clj ... #2019-04-09 17:24:22 rschmukler I don't want to overwrite them outside of the project, and I typically invoke clojure via cider #2019-04-09 17:24:34 rschmukler so, adding project specific environment variables to my emacs config feels quite dirty #2019-04-09 17:24:58 rschmukler (and requires all engineers on the project solve it on their own) #2019-04-09 17:33:12 rschmukler I was able to solve it via dir-locals w/ emacs, for anyone interested... #2019-04-09 17:33:14 rschmukler ((nil . ((eval . (setenv "LD_PRELOAD" "libcurl.so.3")))))
#2019-04-09 17:55:10 alexmiller on clj command line, you can use prefix -J to pass env vars - like -J-Dmy.prop=val #2019-04-09 17:56:15 alexmiller or you can put those in an alias in deps.edn under :jvm-opts #2019-04-09 17:56:52 alexmiller {:aliases
{:foo {:jvm-opts ["-Dmy.prop=val"]}}} #2019-04-09 17:57:04 alexmiller clj -A:foo #2019-04-09 17:57:14 rschmukler Is there a default equivalent? #2019-04-09 17:57:33 alexmiller no, but that has been requested multiple times and I expect will be added when I next get around to it #2019-04-09 17:57:37 alexmiller there's a ticket for it #2019-04-09 17:58:03 rschmukler Cool cool. It's completely understandable that there wouldn't be one (yet, or even ever). Thanks for the response! #2019-04-09 17:58:54 alexmiller oh, you're asking about env vars #2019-04-09 17:59:00 alexmiller I'm talking about jvm opts #2019-04-09 17:59:05 alexmiller those are different things #2019-04-09 17:59:21 alexmiller sorry for the confusion #2019-04-09 17:59:33 alexmiller I consider that out of scope for clj right now #2019-04-09 18:00:08 rschmukler Makes complete sense. Frankly, I think it probably introduces more nasty things than it's worth (eg. people using it as a config store for default environment variables, etc)... This is definitely an edge-case #2019-04-09 18:00:27 alexmiller yeah, I'll resist adding that as long as I can :) #2019-04-09 18:00:36 rschmukler 👍 I think that's a good call #2019-04-09 18:02:09 seancorfield That begs an interesting question: when folks are providing "environment" to processes, do they a) use only environment variables b) use only JVM properties or c) use a combination with one overriding the other? And, for c) which overrides which? #2019-04-09 18:04:02 rschmukler @seancorfield As someone who came to Clojure (and the JVM) after heavily integrating docker + kubernetes, I can tell you I use environment variables for almost everything, and, if possible, don't intend to ever use or learn JVM properties #2019-04-09 18:06:09 rschmukler I think it's mostly that containers (and golang especially) harbored in a lot of workflow / tooling around the idea of using environment variables, and it's something that can apply to outside any one run-time. ie. I can write a service and guarantee environment vars as a config option... If I switch away from JVM for any reason, I now need to provide a config migration story too #2019-04-09 18:07:19 rschmukler regarding (c) - I would expect environment variables to be the last authority... but I think that's likely because of the esteem that I hold them in above #2019-04-09 18:09:33 alexmiller usually in an app I load config from an edn file, but allow override via env, and don't use jvm props at all #2019-04-09 18:10:28 alexmiller I find that gives you a lot of flexibility and options for local dev, CI, prod #2019-04-09 18:10:44 alexmiller but then I don't work on many apps :) #2019-04-09 18:11:16 rschmukler @alexmiller that's exactly what I've done as well. I will sometimes have multiple configs corresponding to an environment (`prod.edn`, dev.edn, test.edn) which might overwrite certain keys in config.edn. I then use ENV to determine which env-specific config to use, while still allowing overwrites for specific vars via environment variables #2019-04-09 18:12:04 rschmukler Elixir also does something similar to the above #2019-04-09 18:16:12 alexmiller if you're willing to ignore the ENV stuff above and use only a stack of appropriately created config files #2019-04-09 18:17:07 rschmukler Brilliant! Similarly, duct does EDN-based config quite well. The docs for it are a bit lacking... But they let you do coercion and environment variables directly via EDN reader macros #2019-04-09 18:18:11 alexmiller yeah, I like duct too from what I recall #2019-04-09 18:18:21 alexmiller I've used environ quite a bit, but can't say I love it #2019-04-09 18:20:06 rschmukler Duct + Integrant have quickly become my favorite way to build / prototype applications. I feel like I'm building an arsenal of components that I can use to build almost any system that I have seen in my career. They integrate the config map w/ spec so you get good error messages, while still having a single map that shows the entire system. It's basically stuartsierra's component extended to a map / graph, rather than relying on manual instantiation from the programmer #2019-04-09 18:20:59 alexmiller yeah, weavejester is great at maximizing the Clojure Way ™ #2019-04-09 18:21:12 rschmukler trade mark #2019-04-09 18:21:15 rschmukler 😉 #2019-04-09 20:01:34 jcf Hi all. I'm looking into deploying a small toy project to Heroku inside a Docker container and I'm running into an interesting problem with my use of tools.deps.
When I build the Docker container locally I can see all the dependencies being downloaded. When I push the container to Heroku and run the thing the same dependencies are installed all over again, which takes so long Heroku kill my container and the app fails to start.
Is there anything container-specific to look out for with tools.deps? My first guess was there was a missing Maven repo somewhere but when I run the container locally all the dependencies are there. Naturally I'm now wondering if Heroku do something different on their platform that might prevent tools.deps from noticing the Maven repo. #2019-04-09 20:03:08 jcf I'm running clojure -Srepro -e '(println "Dependencies installed.")' to install things in my Dockerfile, and then I run the app with exec clojure -Srepro -A:measure -m foo.bar.main. The measure alias only adds a single JVM opt. #2019-04-09 20:03:45 jcf :measure {:jvm-opts ["-Dio.pedestal.log.defaultMetricsRecorder=foo.bar.report/reporter"]}
#2019-04-09 20:05:03 jcf I can reproduce the artefact-download behaviour if I run the container on Docker and invoke my CMD by hand. #2019-04-09 20:05:47 dominicm Is it the same user? #2019-04-09 20:06:29 jcf And I think I might have cracked it. It looks like ~/.m2 is /app/.m2/repository when I run the container but the Docker build step will install things in /root/.m2. #2019-04-09 20:06:41 jcf @dominicm I think you've cracked it too! #2019-04-09 20:07:39 jcf I'm guessing Heroku run things as some other user, which messes with $HOME. I wonder if I can tell tools.deps where to put the Maven repo with the Docker invocations only. Don't really want to mess with that in the project deps.edn as it's kinda up to the dev where they want to keep Maven artefacts. #2019-04-09 20:08:08 jcf Does :maven/local-repo not work in an alias? 🙂 #2019-04-09 20:09:26 dominicm @jcf try -Sdeps perhaps? But remember permissions. #2019-04-09 20:10:30 jcf I'll give that a stab. #2019-04-09 20:20:56 jcf I'm giving -Dmaven.local.repo=/app/.m2 a try. Let's see if that does the trick. #docker 🙂 #2019-04-09 20:24:00 jcf That didn't work. Because it's maven.repo.local. 😕 #2019-04-09 20:24:14 alexmiller you can set it in deps.edn #2019-04-09 20:24:34 alexmiller :mvn/local-repo "/app/m2" as a root level entry #2019-04-09 20:24:46 alexmiller or whatever you need #2019-04-09 20:25:07 jcf @alexmiller I was hoping to only specify that for the Docker side of things to enable the default behaviour for local development. #2019-04-09 20:25:07 alexmiller that's not something you can set in an alias #2019-04-09 20:25:33 alexmiller but you could append by using clj -Sdeps '{:mvn/local-repo "..."}' #2019-04-09 20:26:05 dominicm That's what I was trying to say ^^ but with the brevity that mobile prefers :) #2019-04-09 20:26:35 jcf I tried that originally and the running container still installed everything. I'll bring back the -Sdeps ... approach and poke around inside the container to see if the user in the running container maybe can't access the :mvn/local-repo. #2019-04-09 20:39:47 jcf The -Sdeps {:mvn/local-repo "/app/.m2"} did the trick! Just had to blow away the container on Heroku and push it up again.
Now I need to make sure we stick within allocated memory and start up fast enough despite not installing deps. Might have to uberjar yet.
Thanks for the help, @dominicm @alexmiller! #2019-04-09 21:09:23 alexmiller np #2019-04-11 06:17:00 borkdude if I check some files into a resources folder in my git repo, will someone who uses my repo as a git dep be able to see these resources? I remember there was an issue with this in tools.deps? #2019-04-11 06:17:07 borkdude or was this only when you’re using it as a local/root? #2019-04-11 06:25:11 borkdude after some testing, both seem to work #2019-04-11 06:25:25 borkdude I remember what the issue was with this. maybe a caching issue #2019-04-11 15:00:28 borkdude with leiningen it seems that lib/tools.jar is automatically on the classpath. with tools.deps it doesn’t seem so. what’s the best way to do it without using a cmd line like:
clj -Sdeps "{:deps {tools.jar {:local/root \"$JAVA_HOME/lib/tools.jar\"}}}"
#2019-04-11 15:12:56 alexmiller you should not have to do this #2019-04-11 15:13:24 alexmiller it should automatically be included if you are using the JDK (not the JRE) afaik #2019-04-11 15:13:32 alexmiller are you accidentally using the JRE? #2019-04-11 15:16:33 alexmiller might also be some differences from Java 8 vs Java 9+ (post-module system) in the jdk file structure. I'm not sure there even is a tools.jar anymore? #2019-04-11 15:18:54 alexmiller oh yeah, as of java 9, tools.jar does not exist anymore #2019-04-11 15:19:14 alexmiller needing to include it directly is weird, and probably wrong #2019-04-11 15:30:09 ghadi if the original question was about resources, as long as that directory is listed in the :paths on the dependent repo, then the depending repo will be able to see it #2019-04-11 15:31:18 borkdude I’m using java 8 #2019-04-11 15:31:49 borkdude a quick test if it works in any of your REPLs:
(import 'com.sun.tools.javadoc.Main)
#2019-04-11 15:32:26 borkdude $ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home
#2019-04-11 15:32:31 alexmiller that stuff is dead #2019-04-11 15:32:58 borkdude is there something newer I should use? #2019-04-11 15:33:15 alexmiller yes - javax.tools.DocumentationTool #2019-04-11 15:34:03 alexmiller I don't know the coords for it, but you should just be using it as normal Maven artifact #2019-04-11 15:34:46 borkdude that seems to work out of the box:
$ clj
Clojure 1.10.0
user=> (import 'javax.tools.DocumentationTool)
javax.tools.DocumentationTool
#2019-04-11 15:36:56 alexmiller prob doesn't in newer jdks #2019-04-11 15:37:13 alexmiller actually, it does on Java 11 #2019-04-11 15:37:49 borkdude so I can do something like this as with the older Doclet API? I want to generate EDN #2019-04-11 15:38:03 alexmiller I believe it is the replacement for all that old sun stuff, yes #2019-04-11 15:38:23 alexmiller I'm sure there are buckets of docs to read about it, I've never used it #2019-04-11 15:38:32 borkdude thanks #2019-04-11 16:03:59 borkdude It seems I do need the “dead” Doclet class if I’m using Java 8, since the jdk.javadoc stuff is only on Java 9 #2019-04-11 16:24:24 borkdude so I’m seeing this in the REPL:
(System/getProperty "java.home")
"/Library/Java/JavaVirtualMachines/jdk1.8.0_191.jdk/Contents/Home/jre"
and the same with leiningen. But tools.jar is in ../jdk/lib/tools.jar. #2019-04-11 16:44:40 borkdude I wonder that if I’m going to migrate to JDK 11 if my code will still be able to run for Java 8 users if I’m going to use those new classes, so I’m hesitant about it #2019-04-11 16:54:44 alexmiller it should be fine. that "new" doclet stuff has been around since at least java 7 #2019-04-11 16:55:26 alexmiller seems better than building on something that has been pronounced dead and unsupported #2019-04-11 16:56:21 borkdude but the new doclet stuff is only there since Java 9, or maybe I’m not finding the right stuff on google… #2019-04-11 16:57:01 borkdude OK yeah, I’m going to use that. #2019-04-11 17:02:11 dominicm What are you using doclet for? #2019-04-11 17:07:30 dominicm Oh, it's for reading docs in java. I see. #2019-04-11 17:07:41 dominicm I thought it was for generating them. #2019-04-11 17:05:39 alexmiller everything com.sun is on the removal path #2019-04-11 17:07:56 ghadi make sure you read the migration parts of ^ #2019-04-11 17:08:00 ghadi @borkdude #2019-04-11 17:43:40 borkdude @ghadi thanks! #2019-04-11 17:44:26 borkdude Old type: SeeTag
New type: com.sun.source.doctree.LinkTree
com.sun.source.doctree.SeeTree
Hmm.. #2019-04-11 17:46:48 borkdude Crap, it does seem that this new stuff is only available since 9 #2019-04-11 17:47:33 borkdude Anyway, thanks for the help. #2019-04-11 17:51:55 noprompt Do we have tool for releasing to Clojars in deps land? #2019-04-11 17:52:10 borkdude There are a bunch of these tools on the wiki #2019-04-11 17:52:11 noprompt I’m still lein deploy clojars. 😐 #2019-04-11 17:52:16 borkdude Me too. #2019-04-11 17:52:46 noprompt I reckon if any of those were as convenient as that you’d have mentioned it. 🙂 #2019-04-11 17:53:50 borkdude Haven’t tried them yet #2019-04-11 17:54:42 noprompt I just want a microwave. #2019-04-11 17:54:59 noprompt A jar cooking microwave. #2019-04-11 17:55:10 borkdude depstar is one that I’ve heard of #2019-04-11 17:55:24 alexmiller you can call lein fns from clj #2019-04-11 17:59:00 alexmiller I added that to the wiki page :) #2019-04-11 19:28:28 mpenet I'd still favor pack/depstar #2019-04-11 19:29:36 mpenet I guess it could be improved, the concept is quite interesting #2019-04-11 17:58:48 noprompt Cool. I’ll check that out. pack looks pretty nice. The Juxt folks tend to make decent things. #2019-04-11 20:50:41 dominicm I appreciate the complement. Let me know if you have any issues. #2019-04-12 09:55:07 sooheon Does the capsule method of uberjaring not support AOT compilation? #2019-04-12 10:06:50 dominicm It works with AOT'd assets #2019-04-12 10:06:57 dominicm Why do you think it doesn't? #2019-04-12 10:10:38 sooheon I’m coming from lein uberjar where you explicitly give it a ns to AOT compile. Also saw that Capsule main didn’t care about :gen-class. Do you mean I should AOT compile with something else, and provide the output of that as extra-paths? #2019-04-12 10:11:45 dominicm If you really want to AOT, yeah, I just use clojure -e #2019-04-12 10:12:28 sooheon Ok thanks #2019-04-11 17:58:51 borkdude (it’s not an off the shelve solution) #2019-04-11 18:13:16 martinklepsch I found that there's a thing for every step of the process but none that puts all those bits together sadly #2019-04-11 18:50:09 seancorfield We're using (my fork of) depstar at work for building uberjars. I'm also using it to build "thin" jars for deploying to Clojars (and I use mvn deploy for that, so I run clj -Spom, clj -A:uberjar (alias for depstar's uberjar entry point), and then mvn deploy:deploy-file ...) #2019-04-11 18:50:28 ghadi ^ @noprompt #2019-04-11 18:50:34 ghadi that will get you going #2019-04-11 18:51:10 seancorfield (well, I have a little shell script that combines that -- and also updated the pom.xml with the current SHA of the project so http://clojars.org can point to the commit on GitHub) #2019-04-11 18:53:02 noprompt Nice. I’ll give this a try. #2019-04-11 19:22:21 borkdude Darn.
> This class must contain the start(Root) method.
This is in Java 11 documentation for javadoc which is also used by those new ToolProvider thingies.
But the start(Root) method is from the ancient com.sun code. So this buys me nothing… The documentation for the new Doclet interface doesn’t even mention this though.
https://docs.oracle.com/en/java/javase/11/tools/javadoc.html #2019-04-12 00:42:10 cfleming Is anyone using deps with polyglot JVM projects, e.g. containing Java/Kotlin/etc? #2019-04-12 00:42:30 cfleming I’m interested in recommendations for approaches if so. #2019-04-12 00:43:31 cfleming I see there are some lightweight build tool options listed in the wiki but I don’t know what the state of any of them is. #2019-04-12 01:20:22 alexmiller I probably wouldn’t use deps as the controller for that, I’d use it under make, or whatever #2019-04-12 11:52:33 sundbp Can’t recall if I brought this up before, but can’t find any traces (tickets etc). So we have -Sdeps to add “extra stuff”. For my purposes it’d be very useful to have the equivalent to add extra deps.edn files to be merged in, e.g. -Sdepsfile path/to/extra/deps.edn. As it is logically the same as -Sdeps "$(< path/to/extra/deps.edn)", just packaged in a different way - would there be any pushback to adding such an option? @alexmiller #2019-04-12 12:42:25 alexmiller There is a ticket for this and I’m not opposed to it, but am still thinking about various aspects #2019-04-12 12:56:19 orestis Wait, -Sdeps adds extra stuff? Can it be combined with -A then? #2019-04-12 13:46:30 dominicm It can, yeah. #2019-04-12 13:47:39 dominicm To make sure I don't clobber things, I use a namespaced alias name in scripts, sometimes with generated extra paths (e.g. In tmp) #2019-04-12 13:47:49 alexmiller -Sdeps adds an additional "deps.edn" on the end of the merge chain #2019-04-12 14:16:17 orestis Ohhh that’s so nice. #2019-04-12 19:49:12 sundbp @alexmiller great to hear. #2019-04-13 08:19:15 borkdude if I make an alias, I should be getting the extra paths of that alias when doing -R:the-alias right? #2019-04-13 08:19:30 borkdude somehow it’s not working for me, but it is working with the -A option #2019-04-13 08:22:29 borkdude oh, -C… 🙂 #2019-04-15 20:35:31 borkdude is there a way to let tools.deps print only the paths that you specified in your deps.edn file and not the entire classpath? #2019-04-15 20:37:30 alexmiller No - why would that be useful? #2019-04-15 20:37:41 borkdude linting 🙂 #2019-04-15 20:38:11 alexmiller Why? #2019-04-15 20:38:50 borkdude why linting you mean? #2019-04-15 20:40:04 borkdude one use case: I want to lint the entire classpath, but only report errors from my own code. but it’s not too hard to do some manipulation on the output of -Spath or hard-code it. the amount of paths if fairly limited. #2019-04-15 20:45:21 borkdude to answer your question more generally: it could in theory be useful to list all the files from your own paths for whatever reason #2019-04-15 20:45:45 borkdude but I guess not high priority for now #2019-04-16 05:54:30 orestis Shouldn’t be that complicated to read the file and get the paths out, right? #2019-04-17 11:54:34 orestis Is there an invocation of Clojure that will give the current version number? #2019-04-17 11:56:23 mpenet *clojure-version* I think #2019-04-17 11:56:48 mpenet *clojure-version*
{:major 1, :minor 10, :incremental 0, :qualifier nil}
#2019-04-17 12:03:47 orestis Ah, sorry - I meant about the Clojure CLI tools #2019-04-17 12:04:21 orestis When installing in Linux, I don’t want to download curl -O every time if it’s already present and at the correct version. #2019-04-17 12:08:26 dominicm clojure -Sdescribe? #2019-04-17 12:10:32 dominicm ❯ clojure -Sdescribe | grep -o ':version ".*"' | sed 's/:version //' | sed 's/"//g' #2019-04-17 12:14:08 orestis Fantastic! Thanks 🙂 #2019-04-17 12:38:33 orestis OK, I’m having trouble running the Clojure CLI tools in Elastic Beanstalk, and the AWS support this time isn’t very helpful. #2019-04-17 12:38:58 orestis In my logs, I see this error: mkdir: cannot create directory '/.clojure': Permission denied #2019-04-17 12:39:17 orestis Which indicates to me that something is trying create a .clojure directory at the root of the filesystem, which is weird. #2019-04-17 12:42:10 dominicm Are you installing during user data? #2019-04-17 12:42:36 dominicm Does -Sdescribe try and create .clojure unconditionally? #2019-04-17 12:48:38 orestis It seems so — it’s a mess because Elastic Beanstalk is just duct tape under the hood. #2019-04-17 12:48:58 orestis So you run your Java command with ruby foreman and supervisord… #2019-04-17 12:50:06 orestis Got it, they were setting HOME=/tmp whereas the user actually running the clojure command didn’t have permissions there. #2019-04-17 12:50:25 orestis Setting CLJ_CONFIG to /home/webapp/.clojure seemed to have fixed the problem. #2019-04-17 12:59:03 alexmiller there's a ticket about working in read-only fs btw #2019-04-17 13:08:50 orestis .ebextensions/setup.config file that works: #2019-04-17 13:08:54 orestis option_settings:
- option_name: CLJ_CONFIG
value: /home/webapp/.clojure
commands:
01_install_clojure:
command: |
curl -O
chmod +x linux-install-1.10.0.442.sh
sudo ./linux-install-1.10.0.442.sh
#2019-04-17 13:12:36 alexmiller currently, clj does need some writable path to work as it always drives at least the cp through the cached cp file #2019-04-17 13:17:44 orestis The project .cpcache works fine, it’s the $CLJ_CONFIG that was giving me trouble… #2019-04-17 18:19:00 tanzoniteblack is there an equivalent of lein's :init-nsconfig, where you specify which namespace to load and switch into by default when starting a repl? #2019-04-17 18:37:20 dominicm -e "(in-ns 'foo)" #2019-04-17 21:15:36 kenny Any ideas why I'd hit this? I'm running on a CI that has permission to git clone via HTTPS.
Error building classpath. ...: Authentication is required but no CredentialsProvider has been registered
#2019-04-17 21:17:40 kenny Shoot. Looking at tools.gitlibs, it seems like it may not support cloning private repos via HTTPS 😞 #2019-04-17 21:22:44 ghadi It doesn't. I've worked on that. #2019-04-17 21:22:56 ghadi Have two different approaches for tackling that. #2019-04-17 21:25:03 kenny Ah, I see TDEPS-91 and TDEPS-33. I assume there's no workaround for this right now? #2019-04-17 21:27:24 kenny ... if you could override the tools-deps version the CLI uses. #2019-04-17 23:22:37 ghadi @kenny GitHub or otherwise? #2019-04-17 23:23:06 kenny Where my git deps are coming from? #2019-04-17 23:39:48 alexmiller you can override it if you just hack your clojure script :) #2019-04-17 23:40:30 alexmiller just prepend the source dir to tools_cp #2019-04-18 00:00:08 kenny Interesting... Thinking I'll attempt the path of adding an SSH key to the CI. If too painful, will try the clojure script hacking. #2019-04-18 08:34:45 jeroenvandijk @kenny You can also play with gitconfig and for instance switch between ssh and https without code changes # [url ""]
# insteadOf =
#2019-04-18 08:34:59 jeroenvandijk part of my ~/.gitconfig #2019-04-18 14:42:57 kenny @jeroenvandijk But this won't let tools-deps support https auth, correct? #2019-04-18 14:45:06 jeroenvandijk i did only the opposite as the https auth for github is limited #2019-04-18 14:45:48 jeroenvandijk we're using circleci and they even have a feature to add an ssh key of another user (over a deploy key) to the project automatically #2019-04-18 19:09:02 dominicm @alexmiller what do you think about gitlibs making clones read only so that they aren't accidentally modified by user error? #2019-04-18 19:11:19 alexmiller you mean the working tree I presume #2019-04-18 19:11:30 alexmiller seems reasonable #2019-04-18 19:12:45 ghadi there's no such thing #2019-04-18 19:13:20 ghadi readonly directory means you cannot add or remove to it. modifying an existing file is still fair play #2019-04-18 19:13:33 ghadi you'd have to ensure that every file was r/o too #2019-04-18 19:14:25 alexmiller I assumed that's what we were talking about #2019-04-18 19:14:51 dominicm I'll lodge a JIRA #2019-04-18 19:15:09 alexmiller the dirs under ~/.gitlibs/libs are sha-pegged and shouldn't change #2019-04-18 19:16:03 alexmiller does changing those properties affect whether it looks like the correct working tree to git? #2019-04-18 19:17:13 ghadi huh? #2019-04-18 19:19:34 alexmiller I guess I'm asking if file attributes affect the sha, but looks like no #2019-04-18 22:00:04 dominicm Some do, like +x #2019-04-25 08:56:00 jeroenvandijk Would it be a good idea to add the :inclusions to a maven entry? I'm updating the aws-java-sdk library and i'm really only interested in a few of it's dependencies, but it is exploding #2019-04-25 10:00:50 delaguardo you have :exclusions key defined for maven entry. And it has much less abstract (or opinionated) meaning than :inclusions #2019-04-25 10:06:11 jeroenvandijk Yep. So :exclusions would become unreasonably big with aws-java-sk therefore whitelisting seems like a better approach. Especially since an update means new exclusions #2019-04-25 10:09:49 delaguardo as far as I understand aws-java-sdk is just a collection of packages grouped bu functionality. If you are interested only in one particular package - add just this package and do not add aws-java-sdk #2019-04-25 10:11:14 delaguardo like this {com.amazonaws/aws-java-sdk-s3 {:mvn/version "1.11.539"}} #2019-04-25 10:40:17 jeroenvandijk Ok i guess you have never experienced this problem. I have #2019-04-25 10:40:24 jeroenvandijk Thanks for trying to help though #2019-04-25 11:06:27 delaguardo You might be surprised) maybe you can describe you pain more? it could be helpful for anyone how is willing to help you #2019-04-25 11:20:56 jeroenvandijk e.g. when I try to use Amazonica (`[amazonica "0.3.142"]`) for specific resources I would need to know what to exclude whereas I actually know what to include if I just want to do some s3 things #2019-04-25 11:46:41 delaguardo Main idea - exclude everything, then add parts you know #2019-04-25 12:08:01 jeroenvandijk > Main idea - exclude everything, then add parts you know
Yeah I think you are right. Maybe I was just being impatient #2019-04-25 12:31:37 alexmiller There are split out versions of the stuff under aws-java-sdk #2019-04-25 12:42:07 jeroenvandijk thank you, i guess i was just being impatient. #2019-04-25 14:31:36 unbalanced so I converted the project.clj to deps.edn for starters #2019-04-25 14:32:04 unbalanced #2019-04-25 14:32:13 unbalanced the directory structure is as follows: #2019-04-25 14:32:48 unbalanced #2019-04-25 14:33:06 unbalanced the problem comes when running the tests, the Java classes do not seem to be importing properly #2019-04-25 14:33:49 unbalanced #2019-04-25 14:34:00 ghadi .java files need to be compiled into .class files #2019-04-25 14:34:06 ghadi they cannot be used raw #2019-04-25 14:34:21 unbalanced ahh interesting #2019-04-25 14:34:46 ghadi I would suggest separating the java part of the project into a different artifact, and then including that in the clojure deps.edn project #2019-04-25 14:35:59 unbalanced I think that's a good suggestion. Do you include it with {:local/root "../local-path"} as one would with a clojure project? #2019-04-25 14:36:12 unbalanced or is there a different way to include local java? #2019-04-25 14:36:21 unbalanced or should I upload it to maven or something? #2019-04-25 14:36:43 ghadi upload to maven or clojars #2019-04-25 14:37:02 unbalanced you can upload java to clojars? #2019-04-25 14:37:17 ghadi i think so #2019-04-25 14:37:22 ghadi "it's just a jar" #2019-04-25 14:37:28 unbalanced cool... will look into it. Ah true story #2019-04-25 15:08:16 unbalanced are you sure you can't use .java files without compiling first...? I thought there was some kind of implicit compilation as long as class path was set up properly #2019-04-25 15:08:30 unbalanced because it worked fine for leiningen #2019-04-25 15:08:40 unbalanced via the key :java-source-paths ["jresp/src/main/java"] #2019-04-25 15:10:56 ghadi yeah, I'm sure #2019-04-25 15:11:02 ghadi lein calls javac #2019-04-25 15:15:13 unbalanced ah so that's fundamentally different for the deps.edn approach #2019-04-25 15:54:26 alexmiller yes #2019-04-25 16:33:23 seancorfield You can always use the -e option to shell out to a call to javac 🙂 #2019-04-25 16:33:35 seancorfield (no, don't, it's horrible... but it does work 🙂 ) #2019-04-30 09:33:55 yuhan Is there any way to use a lein plugin in a tools.deps project? E.g. Codox, lein-ns-dep-graph #2019-04-30 09:57:56 delaguardo @qythium most of lein plugins is just a normal projects. You can include them in your dependencies list and execute their -main function if exists or make your own custom -main. E.g. https://github.com/weavejester/codox/blob/master/codox/src/codox/main.clj#L116 there is main function to generate codox documentation. It requires some arguments that you have to assemble by yourself but there is nothing impossible.
clj -Sdeps '{:deps {codox {:mvn/version "0.10.6"}}}' -e "(do (require 'codox.main) (codox.main/generate-docs))"
Oneliner as a prove of concept #2019-04-30 16:50:27 seancorfield And remember, if you're on Clojure 1.10, you can use requiring-resolve so that becomes -e "((requiring-resolve 'codox.main/generate-docs))" #2019-04-30 10:08:52 yuhan wow, I can't believe that just worked out of the box 😮 #2019-04-30 10:10:10 yuhan fully formatted HTML docs of all the namespaces in a large project with zero setup, how awesome is that #2019-04-30 10:10:16 yuhan thanks! #2019-04-30 10:32:06 Christian Johansen when changing the sha on git deps, I frequently run into a variation of this error: #2019-04-30 10:32:07 Christian Johansen Cloning:
#2019-04-30 10:32:16 Christian Johansen is this a known problem, or should I report it in jira? #2019-04-30 10:34:01 dominicm I'm not aware of any similar reports in this channel, nor have I encountered it itself. #2019-04-30 10:34:45 Christian Johansen hmm, I wonder if it could be a false message. I removed the lib from ~/.gitlibs/_repos, and then I see that it errors on auth, as I haven't sun ssh-add after a recent reboot #2019-04-30 10:34:59 Christian Johansen so maybe that error above also was really caused by an authentication failure? #2019-04-30 10:41:04 dominicm Maybe. Could be that there's more to be done with catching it correctly too though. #2019-04-30 11:43:42 alexmiller I haven’t seen anything like this from anyone #2019-04-30 13:02:24 Christian Johansen ok. I'll see if I can reproduce it and if so, report #2019-05-01 07:17:32 luposlip deps.edn newbie here: Anyone's got some intelligent input to the following issue, where a working lein based project, using a java library hosted on jitpack, doesn't work when used via deps.edn (dependency tree doesn't get resolved)?
https://github.com/luposlip/json-schema/issues/1 #2019-05-01 07:44:44 dominicm @luposlip you need to put jitpack in your :mvn/repos #2019-05-01 07:47:24 luposlip even when it's already referred to by the json-schema library?
The user wants to use json-schema, which already refers to jitpack, from a project using deps.edn. #2019-05-01 07:59:35 dominicm I'm not sure that deps can pick those up, there might be a jira. #2019-05-01 08:01:48 luposlip Hmm.. When a library is packaged and deployed to clojars it normally works out of the box when using it from another lein project, regardless of the dependencies and where they are stored.
If this is not the case with clj + deps.edn it seems like a showstopper to me. #2019-05-01 08:01:53 luposlip Ahh, thanks for the link! #2019-05-01 08:02:06 luposlip I'll follow that jira! 🙂 #2019-05-01 08:02:27 dominicm Your vote will help with setting priority #2019-05-01 08:03:55 luposlip I will upvote it! #2019-05-01 23:02:28 kszabo hey, moving from boot to tools-deps + kaocha, which of the many uberjar building tools do you recommend to replace boot’s capabilities? I see several but see no consensus from the community on which is the most stable / can expect future maintenance. #2019-05-01 23:02:28 kszabo hey, moving from boot to tools-deps + kaocha, which of the many uberjar building tools do you recommend to replace boot’s capabilities? I see several but see no consensus from the community on which is the most stable / can expect future maintenance. #2019-05-01 23:43:00 kszabo thanks, but I just noticed that it doesn’t do AOT, guess I’ll need to add that to the list of required features so that the build tooling change does not change the produced artifact #2019-05-01 23:56:35 kszabo It seems badigeon will fit our needs, thanks! #2019-05-01 23:58:11 seancorfield AOT is evil 🙂 #2019-05-02 00:04:06 kenny Yep - we stopped AOT'ing our code when we moved over from Boot. Too many headaches. #2019-05-02 00:07:32 seancorfield We stopped AOT'ing long before that 🙂 Hence depstar does exactly as much as we need: build a JAR from the classpath. In production we do java -cp /path/to/the.jar clojure.main -m our.entry.point #2019-05-02 01:01:35 donaldball What is the current reason to consider AOT evil? We’ve just started using it for our service uberjars to shave a modest but meaningful time off of startup time. #2019-05-02 03:00:04 seancorfield If you're using it as the very last stage, to produce an uberjar, and you really need it to improve startup time, that's "fine". But the Leiningen default where the main namespace is :gen-class'd and AOT is the default in project.clj just gets beginnners into a mess -- and it is completely unnecessary. AOT overall is completely unnecessary -- unless you really, really care about startup time for a final production artifact. #2019-05-02 03:01:20 seancorfield AOT's transitive compilation issues and the problems you can run into when you end up with a mix of .class and .clj versions of the same file can be hell to debug. Much better to just avoid all of that, unless you know what you're doing and you see AOT as purely an end-of-the-line optimization. #2019-05-02 03:01:53 seancorfield The biggest overhead for us at startup time is New Relic's Java agent instrumenting classes (since Clojure generates so many). #2019-05-02 03:02:15 seancorfield And since nearly all of our processes are long-running, we just don't care about startup time. #2019-05-02 15:03:00 kenny I could see it mattering more if you need to scale your service quickly due to large demand spikes. #2019-05-02 17:19:28 kszabo In the end I took your advice and did not AOT before uberjaring. Still I stayed with badigeon because it allowed me to exclude some cljs related files I don’t need from my resources, which sadly depstar doesn’t support, having a static exclude function. #2019-05-02 17:24:07 seancorfield Feel free to create an issue for depstar to exclude files -- I just refactored some of that code recently so it'll be easier to extend to support options (although, right now, depstar has no command-line options at all beyond the JAR filename). #2019-05-02 15:56:35 avi 👋 I just searched this channel, the Web, and the Clojure JIRA for anything related to git dependencies that use submodules, but didn’t turn up anything… I’m trying to include a dependency via git, and when I run clojure -Stree it fails with org.eclipse.jgit.api.errors.JGitInternalException: Checkout conflict with file: resources/devtools-protocol which is caused by org.eclipse.jgit.errors.MissingObjectException: Missing unknown 40a14030 (sha truncated) — that’s the sha of the submodule.
Anyway I’m not looking for free troubleshooting, just wondering whether this is a known issue and whether it’s on anyone’s radar? If not I could create a ticket…? #2019-05-02 15:57:01 dominicm I don't think this has come up 🙂 a jira is best I think #2019-05-02 15:57:10 avi alrighty — thanks! #2019-05-03 20:01:50 vlaaad I really want to have add-lib branch into master, dynamic dependency loading removes so much friction! #2019-05-03 20:29:52 dominicm It doesn't need to be on master #2019-05-03 20:41:43 seancorfield @vlaaad you can depend on the SHA for it -- that's what I do. Alex generally keeps that branch up-to-date with master. #2019-05-03 20:56:10 vlaaad I know I can depend on shas, but that branch is 18 commits behind #2019-05-03 20:58:15 vlaaad oh, latest commit was merging master into it, nice #2019-05-03 20:59:40 vlaaad anyway, being unofficial and partially supported is concerning: I want newest stuff as well #2019-05-03 21:19:19 dominicm Development workflows can be a little more alpha, especially something low risk like this. If it doesn't work, you restart the repl instead. #2019-05-04 09:20:57 yuhan how do I add jvm-opts via the clj command? #2019-05-04 09:21:58 yuhan clj -Djdk.attach.allowAttachSelf does not work #2019-05-04 09:28:52 yuhan right, I figured it out - clj -J-Djdk.attach.allowAttachSelf #2019-05-04 09:32:54 yuhan but to add it to my deps.edn file I can't just insert :jvm-opts ["-Djdk.attach.allowAttachSelf"] at the top level? #2019-05-04 10:43:14 alexmiller No, that’s an outstanding issue. You will need to put it in an alias and use the alias #2019-05-06 14:10:18 tianshu I want use -Jopt to add a jar file into classpath. what is the correct syntax looks like? clojure -J-cpjarpath ... seems to be no effect and clojure -J-cp jarpath ... will cause an error. #2019-05-06 14:22:47 dominicm @doglooksgood use -Sdeps #2019-05-06 14:27:07 alexmiller clj builds the classpath so you must use your deps.edn for any modifications to the classpath. in this case, you'll want to use a local jar lib dependency - like my/lib {:local/root "/path/to.jar"} #2019-05-06 14:28:00 alexmiller if you just want to add it sometimes, either put it in an alias with :extra-deps or declare it dynamically on the command line with -Sdeps #2019-05-06 18:43:39 sparkofreason Getting this error running clojureL com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available. This is on WSL running under Windows 10. Seem to recall having this issue before, but I don't remember the fix. git commands work fine, I recall this being something specific with the Java ssh implementation. #2019-05-06 18:51:59 sparkofreason @seancorfield This was encountered while attempting to build an uberjar with depstar. Tried on Windows instead of WSL, where I get Exception in thread "main" java.nio.file.NoSuchFileException: C:\Users\dave\AppData\Local\Temp\uberjar8947452084826452117\uncomplicate\neanderthal\aux.clj, which seems weird, like there was a file in the manifest that didn't get copied. #2019-05-06 18:54:02 seancorfield @dave.dixon If you can repro easily, feel free to create an issue on GitHub with details and I'll take a look -- but I suspect something AOT-related #2019-05-06 22:30:55 seancorfield @dave.dixon Thanks for the repro case. It's definitely a weird one. Files/copy is failing on the target file during the copy which suggests the parent folder is missing (it isn't) or there is a permissions problem (there isn't). I've tried various changes to depstar locally but cannot figure out what the issue is. #2019-05-06 22:32:07 seancorfield Tracing through the JDK source, it seems to hinge on the last Windows filesystem error triggering an exception so I thought maybe it was a sync'ing issue -- but using SYNC or DSYNC for StandardOpenOption made no difference either. #2019-05-06 22:36:29 sparkofreason I cloned and ran with :local/root and it works. 🤔 #2019-05-06 22:40:00 sparkofreason Wait - never mind, I was commenting out the neanderthal dependency 😬 Been that kind of day. #2019-05-06 23:16:23 sparkofreason @seancorfield If I exclude that file explicitly, it "works", in the sense of not throwing that exception. So it seems to be that single aux.clj file that's an issue. #2019-05-06 23:17:02 seancorfield Weird. #2019-05-06 23:46:15 seancorfield And it gets weirder. So I modified depstar so that if a copy operation fails, it tries again, just copying the file to the top of the tree in the temp folders (so uncomplicate/neanderthal/aux.clj copies to thing.clj) and then it "succeeds" and copies the files that follow aux.clj in the Neanderthal JAR. However, I got a weird NoSuchFile exception trying to write test.jar after that (where it shouldn't matter if the file exists or not). I ran touch test.jar and re-ran the command -- and it worked just fine (with aux.clj copied to the top of the tree). #2019-05-07 00:17:47 alexmiller are there two aux.clj files somehow? #2019-05-07 00:17:58 alexmiller such that you're hitting a locked file or something? #2019-05-07 00:18:20 alexmiller jars can contain two files of the same name or a file and a directory of the same name #2019-05-07 00:20:32 seancorfield Nope. Just the one aux.clj #2019-05-07 00:22:51 seancorfield It's really odd because the exception (no such file) comes from a copy operation for the target file -- which doesn't exist because that's the file you're trying to copy to #2019-05-07 00:23:37 seancorfield and it succeeds in copying additional files to that same folder location after failing on this one file. #2019-05-07 00:24:15 seancorfield And you can copy the file to some other location in the tree... #2019-05-07 00:24:47 alexmiller well, file systems are a new fangled technology, wouldn't expect them to be foolproof yet #2019-05-07 00:24:59 alexmiller if only they were statically typed, we wouldn't have these bugs #2019-05-07 00:29:27 seancorfield uncomplicate/neanderthal/internal/device/common.clj
uncomplicate/neanderthal/internal/device/cublas.clj
uncomplicate/neanderthal/internal/device/clblock.clj
uncomplicate/neanderthal/aux.clj
java.nio.file.NoSuchFileException C:\Users\sean\AppData\Local\Temp\uberjar11308132828377820906\uncomplicate\neanderthal\aux.clj
[u n c o m p l i c a t e / n e a n d e r t h a l / a u x . c l j]
[117 110 99 111 109 112 108 105 99 97 116 101 47 110 101 97 110 100 101 114 116 104 97 108 47 97 117 120 46 99 108 106]
java.nio.file.NoSuchFileException C:\Users\sean\AppData\Local\Temp\uberjar11308132828377820906\_random.txt -> C:\Users\sean\AppData\Local\Temp\uberjar11308132828377820906\uncomplicate\neanderthal\aux.clj
failed to copy back over
uncomplicate/neanderthal/block.clj
uncomplicate/neanderthal/linalg.clj
uncomplicate/neanderthal/real.clj
uncomplicate/neanderthal/core.clj
uncomplicate/neanderthal/opencl.clj
uncomplicate/neanderthal/internal/device/vect-math.cl
uncomplicate/neanderthal/internal/device/blas-plus.cl
It successfully copies files before and after. It can copy that one file to _random.txt happily but still can't copy it to the uncomplicate\neanderthal\aux.clj location after that. #2019-05-07 00:44:53 seancorfield I can even copy it to uncomplicate/neanderthal/not.aux.clj ... is a filename of aux.* somehow magical on Windows? #2019-05-07 00:54:14 seancorfield So I bet that is what is happening here. Wow... #2019-05-07 00:55:13 seancorfield PS C:\Users\sean\clojure\depstar-test> touch aux.clj
C:\Program Files (x86)\Gow\bin\touch.exe: cannot touch `aux.clj': No such file or directory
PS C:\Users\sean\clojure\depstar-test> touch con.clj
C:\Program Files (x86)\Gow\bin\touch.exe: setting times of `con.clj': Invalid argument
PS C:\Users\sean\clojure\depstar-test> touch prn.clj
C:\Program Files (x86)\Gow\bin\touch.exe: cannot touch `prn.clj': No such file or directory
PS C:\Users\sean\clojure\depstar-test> touch prn1.clj
PS C:\Users\sean\clojure\depstar-test>
:rolling_on_the_floor_laughing: #2019-05-07 07:25:17 tianshu @dominicm @alexmiller thanks! #2019-05-07 07:25:28 tianshu very clear #2019-05-07 13:04:53 sparkofreason @seancorfield @alexmiller Since I seem to be completely blocked on building on windows, any idea on my WSL issue above? #2019-05-07 13:39:14 mhuebert if one specifies a tools.deps version i deps.edn, is that version somehow used to resolve other deps? #2019-05-07 13:41:59 alexmiller no #2019-05-07 13:42:29 alexmiller there isn't any way "official" way to change the tools.deps version you're using #2019-05-07 13:43:20 alexmiller that said, you can edit your clojure script and modify the tools_cp to point to a different source version of tools.deps (and that's how I test a lot of stuff) #2019-05-07 13:44:31 alexmiller so instead of tools_cp="$install_dir/libexec/clojure-tools-1.10.0.442.jar", something like
tools_cp="/path/to/my/tools.deps.alpha/src/main/clojure:$install_dir/libexec/clojure-tools-1.10.0.442.jar" #2019-05-07 13:45:40 alexmiller eventually I'll provided an env var for that so you can just set the var rather than hack the script #2019-05-07 13:50:50 mhuebert hmm. i’m configuring a circleci build which has some git deps that need the recent local/root changes, currently it installs clojure via the #2019-05-07 13:54:59 alexmiller the local/root changes are all in the latest release - why can't you just use it? #2019-05-07 13:55:14 mhuebert oh, then maybe i have a different problem #2019-05-07 14:02:50 mhuebert maybe something to do with the way circleci sets paths #2019-05-07 14:05:28 mhuebert Manifest type not detected when finding deps for chia/reactive in coordinate #:local{:root "/home/circleci/chia/reactive"}
& there is nothing in the /home/circleci/ dir, except form the main “/project” working directory #2019-05-07 14:06:43 alexmiller yeah, so it can't detect what kind of project that is as there is no manifest file (deps.edn or pom.xml) #2019-05-07 14:07:41 mhuebert so chia is in /home/circleci/.gitlibs #2019-05-07 14:08:36 mhuebert the clj -Spath command is executed in the ~/project/editor dir #2019-05-07 14:08:51 mhuebert which has a deps.edn pointing to the :git dep #2019-05-07 14:09:06 alexmiller both local deps and git deps try to detect their project type by looking for well known files at the project root #2019-05-07 14:09:06 mhuebert the git dep is downloaded #2019-05-07 14:10:08 mhuebert it appears to be finding the root deps.edn at ~/.gitlibs/libs/chia/chia/da6ceb2989b712ab59c654679e1eb7b2e4c6ef02 #2019-05-07 14:11:35 mhuebert or one of the sub-dirs.. have to check #2019-05-07 14:11:40 alexmiller the error you have above indicates something is referencing a local dep at /home/circleci/chia/reactive #2019-05-07 14:12:09 alexmiller that may be the resolved path of a relative dir in the referring deps.edn #2019-05-07 14:13:03 mhuebert the root dep has chia/reactive {:local/root "reactive"}, and some sibling deps have chia/reactive {:local/root "../reactive"} #2019-05-07 14:13:12 mhuebert i have no idea how it is navigating so far up #2019-05-07 14:14:12 alexmiller its based on the working directory #2019-05-07 14:16:20 mhuebert hm i thought that with the recent changes, one could use relative paths within a git dep, and those would be resolved not from the current working directory, but relative to the path into the git dep where the relevant deps.edn is being read? #2019-05-07 14:21:37 alexmiller yes, should work that way. I don't have enough context to understand where things are going awry for you #2019-05-07 14:21:54 mhuebert ok. i will keep digging, thank you for the pointers #2019-05-07 14:22:04 alexmiller clj -Stree might help, not sure #2019-05-07 17:10:10 dwhite44 i have a project that is using a git reference to a private repo. everything works fine locally (mac) but when I try anything on our build server (CentOs), I get “Error building classpath. connector is not available:
com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available”. this happens just by running /usr/local/bin/clojure. If I comment out the dependency to that git repo, it works fine and brings in all the maven dependencies. I can clone that repo without issue so doesn’t seem like it would be a git access issue. any ideas? #2019-05-07 17:57:17 alexmiller it's trying to use the ssh agent, which does not seem to be running #2019-05-07 17:59:36 tanzoniteblack @dwhite44 a number of people in our company have had this issue since we switched to tools-deps. If you're on a mac, make sure you've explicitly run ssh-add -K in the terminal. If that still doesn't work, try removing your ssh config (i.e. mv ~/.ssh/config ~/.ssh/config.bak). Some of the people in our company have had something in their config that jgit chokes on. Never have figured out exactly what #2019-05-07 18:00:36 dwhite44 ok. just not sure how the same repo would clone fine if the ssh agent isn’t running #2019-05-07 18:02:23 alexmiller there are several possible reasons due to the differences between how git finds credentials vs how jgit+jsch does it #2019-05-07 18:02:56 tanzoniteblack ^^^ That has been a major source of headaches #2019-05-07 18:04:02 sparkofreason I have ssh-agent running, added the key, no config file, same error. #2019-05-07 18:13:54 alexmiller yeah, Ghadi Shayban has done a full rewrite of tools.gitlibs that just shells out to git, which has different tradeoffs but at least generally "does what git does" #2019-05-07 19:59:25 seancorfield I just pushed depstar "0.2.0" to clojars -- it's a rewrite to leverage the ZipFileSystem, as a way to get around Windows' reserved filename issues, which means a single copy pass so it's also much faster. Would love feedback from anyone already using depstar. #2019-05-07 20:00:58 seancorfield (although it now produces slightly different ownership/permissions on the resulting JAR file, I think because of the FileSystemProvider approach) #2019-05-09 10:18:05 kszabo hey, how do you manage compilation of Clojure :gen-class files with tools.deps at development time? Do you call clojure.core/compile manually? #2019-05-09 16:52:57 dominicm I do, yeah #2019-05-09 17:53:45 seancorfield {:deps {org.clojure/clojure {:mvn/version "1.10.0"}}
:paths ["classes" "src"]
:aliases
{:compile {:main-opts ["-e" "(compile,'cfml.struct)"]}
:build {:extra-deps {seancorfield/depstar {:mvn/version "RELEASE"}}
:main-opts ["-e" "(compile,'cfml.struct)"
"-m" "hf.depstar.jar" "cfml-interop.jar"]}}}
is an example from one of my projects. I run it with clj -A:compile to get a REPL or clj -A:build to create a JAR for deployment. It duplicates the compile call in both. I could just have it in :compile but then I'd have to remember clj -A:compile:build whenever I wanted to build a JAR. #2019-05-10 09:48:53 kszabo Thanks! We have a bit more complicated project with shadow-cljs + deps.edn with docker images. Since we already had a Makefile to run docker-based integration tests locally, we just added a new target to it, essentially doing what you are doing above with a manual clojure invocation. #2019-05-09 14:51:41 hlolli in leiningen there's a possibility of defining native paths with :native-path, that causes all files on this path to be spit'ed out into the current working directory, which suites well for jna/jni apps using .so files, I'm not seeing this behaviour in tools.deps, is it missing, not implemented or am I missing an argument? #2019-05-10 15:02:55 hlolli very nice, thanks! #2019-05-09 14:59:28 alexmiller Not implemented #2019-05-09 15:02:18 hlolli prima 🙂 no stress, then I'll continue to advise to Overtone users to stick with leiningen until then #2019-05-09 18:41:17 seancorfield (! 1084)-> clj -e '(set! *assert* false)' -r
false
user=> (assert false)
nil
user=> (assert true)
nil
user=> ^D
Thu May 09 11:40:55
(sean)-(jobs:0)-(~/clojure)
(! 1085)-> clj -e '(set! *assert* true)' -r
true
user=> (assert false)
Execution error (AssertionError) at user/eval138 (REPL:1).
Assert failed: false
user=> (assert true)
nil
user=>
#2019-05-09 18:41:44 seancorfield Why "imagine" when you can just "repl"? 🙂 #2019-05-09 18:43:08 borkdude true. I’m just wondering if there was a java property for it #2019-05-09 18:43:39 borkdude I’ll just put an assert in my code and see if it fails, thanks #2019-05-09 18:43:40 seancorfield There's a JVM property for spec asserts. #2019-05-09 18:44:04 seancorfield (but not regular Clojure asserts I believe) #2019-05-09 18:44:13 borkdude yeah, that’s why I imaged there would be one for regular asserts as well #2019-05-09 18:48:03 borkdude works 🙂 #2019-05-09 19:14:08 borkdude :jvm-opts are only supported in aliases, not on the top level right? I don’t see this in the documentation and I remember asking this before #2019-05-09 19:14:36 ghadi right ^ #2019-05-11 10:51:19 olivergeorge Are there any tips for debugging TransportException "Auth fail" errors? I swear I haven't changed anything and I'm suddenly getting this error. I can manually git clone the repo via ssh just fine.
Error building classpath. : Auth fail
org.eclipse.jgit.api.errors.TransportException: : Auth fail
Full stack trace here: https://gist.github.com/olivergeorge/17105466d16021269415ff66423a91ab #2019-05-13 14:37:01 alexmiller can you file a jira on TDEPS? #2019-05-13 15:52:04 timgilbert Hmm, actually looking more closely I think your issue is something different, never mind #2019-05-14 13:47:59 stathissideris when using a git dependency, do I also get access to the resources of the dependency or just the source? #2019-05-14 14:40:38 dominicm @stathissideris what's the distinction? #2019-05-14 14:40:47 dominicm If its in :paths, you get it #2019-05-14 14:41:09 stathissideris @dominicm yeah, I just figured that out by experimenting 🙂 thanks! #2019-05-15 16:32:49 Chris Nevermind! It turned out to be relatively easy to write my own 🙂 #2019-05-15 17:54:48 richiardiandrea A little rusty with Clojure on JVM...if I want to work against a Java code base, do I need to include taarget in order to import classes in the REPL using clojure? #2019-05-15 17:55:33 alexmiller yes, assuming target is the root of your compiled classes #2019-05-15 17:56:11 richiardiandrea ok cool that's what I was missing then, I guess it goes without saying that I then need to compile the java code every time the file changes #2019-05-15 17:59:22 richiardiandrea thank you Alex that works #2019-05-15 20:18:51 holyjak Hello, what do people use to watch for file changes to trigger actions (e.g. clojure.core/compile)? Lein certainly has a watch functionality/plugin but what about clj CLI? Thanks! #2019-05-15 20:20:36 alexmiller as above, clj doesn't cover compilation #2019-05-15 20:20:43 alexmiller it does cover "running programs" #2019-05-15 20:20:47 alexmiller the compiler is a program #2019-05-15 20:20:50 alexmiller watching is a program #2019-05-15 20:21:00 alexmiller compose programs, run with clj if you like #2019-05-15 20:38:30 holyjak thank you, Alex. My question is essentially: Does anyone made a clj-friendly plugin for watching files and executing actions upon change? #2019-05-18 13:42:28 orestis I like hawk for this. #2019-05-15 20:40:33 alexmiller I'm not aware of something specific to tools.deps, but there several watcher type things out there #2019-05-15 20:40:45 alexmiller last time I needed one, I just wrote the 60 lines of java interop to use the one in the jdk #2019-05-15 20:41:22 alexmiller admittedly, it's a weird java api #2019-05-15 20:53:46 souenzzo @holyjak I use (load "core") to load/reload files
But usually it's a "built-in" feature from your IDE/Plugin/Editor/.. #2019-05-15 21:07:41 holyjak well, I need to watch for changes to run compile automatically... #2019-05-15 21:08:57 seancorfield Personally, I don't like watcher-based workflows. I usually find the watch fires when I don't want it to and/or gets into a bad state because it fires while there are still errors in the code. #2019-05-15 21:09:41 seancorfield I'd much rather have an explicit compile command in a (comment ...) form that I can eval from my editor without even worrying about what the terminal is doing. #2019-05-15 21:12:03 seancorfield (also, why do you need to run compile at all? what features are you relying on that require Clojure code to be compiled?) #2019-05-15 22:23:56 holyjak https://github.com/holyjak/clj-concordion - integration with Java Specification by Example test framework that expects me to give it Fixture objects with test-specific methods. So you do `(deffixture "class.Name" [fns2expose as methods]) for each specification document. #2019-05-15 22:24:47 holyjak good points about the downsides of watchers, thank you! #2019-05-15 22:25:12 seancorfield Ah... so you're relying on code gen / gen-class rather than reify for this? #2019-05-15 22:26:00 holyjak exactly ,as there is no interface to reify #2019-05-15 21:10:47 holyjak Wied problem: while this works clj -J-javaagent:bin/springloaded-1.2.5.RELEASE.jar -J-noverify
this fails: ; deps.edn: [...]
:aliases {:agent {:jvm-opts ["-J-javaagent:bin/springloaded-1.2.5.RELEASE.jar" "-J-noverify"]}}
; CLI:
$ clj -O:agent
Unrecognized option: -J-javaagent:bin/springloaded-1.2.5.RELEASE.jar Why? #2019-05-15 21:11:26 seancorfield -J passes options through. They don't start with -J inside :jvm-opts #2019-05-15 21:11:46 seancorfield You want :aliases {:agent {:jvm-opts ["-javaagent:bin/springloaded-1.2.5.RELEASE.jar" "-noverify"]}} #2019-05-15 21:12:03 holyjak oh I see, please ignore 🙂 I have to omit -J #2019-05-15 21:20:34 holyjak Given a .clj file path, what is the simples way of finding out the namespace of the file? I could read the first line and check for #"\(ns (\w+)" but there is surely a cleaner way? #2019-05-15 21:24:48 seancorfield There's no guaranteed way because you can change namespaces inside a file and you can load arbitrary source files into another namespace. #2019-05-15 21:26:19 seancorfield The convention is path/to_the/file.clj => path.to-the.file tho' -- note the _ to - mapping. And you'll need to know what it is _relative to your classpath_ because if src/main is on your classpath, src/main/path/to_the/file.clj is what will get loaded if you try to require path.to-the.file #2019-05-15 21:32:30 holyjak thank you! #2019-05-15 21:40:20 alexmiller tools.namespace is a lib for doing this kind of thing #2019-05-17 18:23:25 sogaiu i have an alias for starting a socket repl in my ~/.clojure/deps.edn -- does anyone have a good way to get the port chosen to be somewhat random and then print that port number to stdout so i know where to connect to? #2019-05-17 19:00:00 lilactown I don’t think you can parameterize over aliases like that, but you could write a small script around it perhaps? #2019-05-17 20:00:48 alexmiller rather than relying on the startup params, you could just write a small .clj that programmatically chose a random port and started the server #2019-05-17 20:00:55 alexmiller and then have the alias run that #2019-05-17 20:01:58 alexmiller with the same params you pass on the cmdline mostly #2019-05-17 20:56:33 sogaiu thanks for the ideas, will consider #2019-05-17 22:07:16 seancorfield As an example, here's what we do at work. #2019-05-18 00:12:05 sogaiu thanks for the sample code! #2019-05-17 22:08:04 seancorfield That also starts Cognitect's REBL if it's on the classpath. And it saves the Socket REPL port in a dot file for next time. #2019-05-17 22:08:15 seancorfield (to sort of match what nREPL tools do) #2019-05-18 12:27:54 holyjak Is there any trick for using deps.edn with a :local/root dependency to make something like "~/.m2/repository/.../some.jar" work without having to hardcode the actual path to the $HOME directory? The beauty of ~/ is that it is quite cross-platform... #2019-05-18 13:15:52 alexmiller If it’s in your m2 repo, then why not use a Maven coord? #2019-05-20 17:33:49 Aaron Cummings Is there a flag for clj to force a refresh from the Maven repos, similar to the -U flag on lein? #2019-05-20 17:50:43 alexmiller currently, no #2019-05-20 17:50:49 alexmiller there is an outstanding ticket for this #2019-05-20 17:51:47 alexmiller one option is to wipe the portion of your ~/.m2/repository as that's where the cache of update times is held #2019-05-20 17:52:13 alexmiller I'd recommend being as selective as possible in deleting parts of ~/.m2/repository #2019-05-20 17:55:19 Aaron Cummings I did work around this with the 'mv /.m2 /.m2.to-be-removed' nuclear option. I'll try to find that ticket and upvote it. #2019-05-20 18:03:20 Aaron Cummings Found it - TDEPS-62 #2019-05-20 20:05:16 Aaron Cummings (not sure if it makes sense to upvote in the old Jira) #2019-05-20 20:12:46 alexmiller well it won't hurt, but also it won't help :) #2019-05-20 20:13:14 alexmiller it's ok, I am aware of the issue and have a plan for it :) #2019-05-20 21:00:31 Aaron Cummings Thanks Alex. #2019-05-21 16:25:27 didibus When using a git repo, can you make it depend on a particulars branch head instead of a specific sha? #2019-05-21 16:25:34 didibus Say I always want the latest commit #2019-05-21 16:26:13 didibus My idea would be to have a release branch, where each commit is a new stable release #2019-05-21 16:26:50 seancorfield @didibus Take a look in my dot-clojure file -- I give an example of how to resolve master HEAD to a SHA in there. #2019-05-21 16:27:15 seancorfield Basically, tools.gitlibs provides tag/branch resolution #2019-05-21 16:29:14 didibus Hum #2019-05-21 16:30:11 didibus I see, but that's basically the same as the user going to github to lookup the latest sha, just instead he can find it from inside a Clojure repl #2019-05-21 16:30:31 didibus Or asking him to always dynamically inject the dependency #2019-05-21 16:31:19 seancorfield You can hide it away behind a bit of code, but yes, it requires a lookup and a dependency load (my example is via add-lib). #2019-05-21 16:31:32 didibus I was hoping for something where I can just say, add this alias and done. Now the user always gets the latest release, so my CLI is pretty much auto-updating. #2019-05-21 16:32:13 didibus Like how mvn/version can be "RELEASE" #2019-05-21 16:32:54 seancorfield Yeah, it would be nice if :sha (or an equivalent) could specify a label to resolve first -- but that goes against repeatability. #2019-05-21 16:33:29 seancorfield And I believe even Maven's RELEASE is considered deprecated? Or at least somewhat bad practice... #2019-05-21 16:35:48 ghadi tags are references, tools.deps only deals in values #2019-05-21 16:35:50 alexmiller using a changing version destroys the ability to cache and select versions #2019-05-21 16:35:58 alexmiller so it's intentionally not supported #2019-05-21 16:36:27 didibus Yes, and I understand it in the Java world. I think Release is deprecated or even removed in Maven 3.x. But in the Clojure world, tools.dels is also used as a minimal package manager for Clojure build tools and CLI apps. For that, repeatability isn't a concern, but auto-update is. #2019-05-21 16:37:07 didibus Your dot-file demonstrate that use case. #2019-05-21 16:38:00 alexmiller auto-update is not a use case tools.deps is trying to cover right now #2019-05-21 16:38:00 didibus Hum... Can't you still cache the sha of the head? Or you mean, the cache won't be as good since it will always need to check the repo for the latest commit sha? #2019-05-21 16:38:33 alexmiller sha's don't change meaning #2019-05-21 16:40:59 didibus The workaround I'm thinking is to wrap my CLI in another Clojure dependency that performs a dynamic add-lib and resolves latest. A kind of get-latest shim. But I kinda hate it #2019-05-21 16:41:49 didibus That one would never need updating (hopefully), and would always just bootstrap my latest #2019-05-21 16:42:30 alexmiller not saying this isn't a use case worth considering, just not one we are trying to cover right now #2019-05-21 16:42:54 didibus Got it #2019-05-21 16:43:32 didibus I can live without it and patiently wait. Just thought I'd see if there was already something of that sort. #2019-05-21 16:44:43 didibus I might be able to create an update command on the cli itself that edits the deps.edn file to its latest sha. That might be cleaner. #2019-05-21 16:49:13 didibus I am very pro for reproducible builds. And now that I'm thinking about it, I can't see good reasons for local deps.edn to not be version specific, even in their version of build tooling, test runner, etc. Since you want everything exactly the same to reproduce. But for the global deps.edn, which is more for installing Clojure utils on your machine, which as I see it, is like a package manager for Clojure apps, such a feature would be great. #2019-05-21 16:50:47 alexmiller Yeah #2019-05-21 16:51:45 alexmiller Although you’d be pretty unhappy if your tool auto updated and broke stuff #2019-05-21 16:53:09 lilactown yeah I like the fact that I can pin the tools to a specific sha. makes troubleshooting much easier #2019-05-21 17:03:41 seancorfield I only use RELEASE for dev tools locally, although I made it the default in new projects created with clj-new -- on the assumption that when you start off, getting the latest versions is fine but you'd change it to a specific version as soon as you start to do any serious work. #2019-05-21 17:04:38 seancorfield (and I didn't want to get into the situation of lein where it often lags way behind Clojure's releases and you need a whole new lein release just to bump the Clojure version in new projects) #2019-05-21 17:08:06 lilactown good point seancorfield #2019-05-21 17:21:38 didibus Hum... Ya. I'm wondering if a tools.deps tool could itself suffice. Like just have a -A:update-latest-deps #2019-05-21 17:22:09 didibus Which would resolve latest maven and git releases and update your tools.deps file. #2019-05-21 17:22:26 didibus So the user is in charge of when to update #2019-05-21 17:22:33 didibus It could even have a rollback feature #2019-05-21 17:22:49 didibus And what not, to make it more resilient in the case of breakage #2019-05-21 17:24:27 didibus Also, another thought that crossed my mind, and I'm just brainstorming at this point. I've noticed my default usage of Clojure cli is -A. Would it make sense then to default all unknown commands to be treated as an alias? #2019-05-21 17:24:57 didibus So clj new bla would be treated as clj -Anew bla #2019-05-21 17:25:57 didibus Like when it's not a valid path to a script #2019-05-21 17:26:35 seancorfield Most likely you would not want all Git deps updated, just certain ones. #2019-05-21 17:27:39 didibus Ya, I'm sure it could be enhanced with some options to exclude or include. Or even made interactive so it asks you Y/n for each one you want to update #2019-05-21 17:26:47 didibus Or to be honest I'd rather have to use say -s for running scripts, and aliases be the no option default #2019-05-21 17:27:35 seancorfield It's kind of important to have the -R/`-C`/`-M`/`-O` alias options tho' right? -A is convenient but kinda lazy. #2019-05-21 17:29:38 didibus Hum.. Ya I was wondering that. I guess I'm very much thinking here about the specific use case where tools.deps is used as an app launcher for Clojure and a build script launcher. #2019-05-21 17:30:36 lilactown I won't speak for alexmiller / et. al. but it's my impression is that they are only interested in solving "resolving dependencies and constructing the classpath" right now #2019-05-21 17:32:24 didibus Ya, I'm kinda just wishy washy throwing some ideas which I haven't put much thought into. So none of this is criticism or even feature request, since I'm not even convinced any of this is a good idea yet. #2019-05-21 17:33:39 didibus It's just that, the more I think about "resolving deps and constructing the classpath" the more I realize that's basically all you need for being a package manager and app launcher for Clojure. #2019-05-21 17:36:12 jmv hi all, i'm trying to get a coworker setup with a clojure project and we are hitting this error
org.eclipse.jgit.api.errors.TransportException:
is this an ssh auth error? #2019-05-21 17:52:04 jmv to follow up, we found the culprit. it was a bad entry in the known hosts file. #2019-05-21 17:38:34 didibus The only real part missing is the idea of a "package". Its technically what an alias is. It's a preconstructed set of dependencies and main entry point which together consists of a working release of an app. But currently, aliases aren't easily shareable. Like you can't publish them, or update them easily. And launching them is a little ackward due to the use of -A. #2019-05-21 17:42:21 seancorfield @didibus The one thing I really "miss" with aliases is the abilty to compose them in deps.edn. Otherwise you either need to duplicate stuff under multiple aliases or the user has to specify multiple aliases on the command line. #2019-05-21 17:43:44 seancorfield (and I do generally use -A out of convenience rather than caring about whether I need -R or -M -- even tho' for things like test-runner, being able to use -R:test vs -R:test -M:test or -A:test would be "better practice" in a way) #2019-05-21 17:44:17 didibus Hum.. I feel maybe it's that aliases are currently used as packages, where really they were meant as ways to optionally add more or less dependencies to your running code, for development purposes mostly. #2019-05-21 17:44:45 didibus Maybe adding something like -P would make more sense for these use cases. #2019-05-21 17:45:16 didibus Where its clear you won't be changing the set of dependencies, but fully using the one defined by it #2019-05-21 17:45:34 seancorfield I don't see the "alias as package" similarity at all... puzzled #2019-05-21 17:46:33 didibus So, I might want to inject a test runner and also have my src on the path and run that. So I create an alias to inject the test runner. That's one use case. #2019-05-21 17:48:00 didibus But maybe I want to be able to run a command line editor implemented in Clojure. So I add an alias to it in my global deps.edn file and then to run it I do clojure -Acljedit bla.txt #2019-05-21 17:49:05 didibus This command installs the cljeditor app on my machine, and launches it. On the next run, the install step is skipped (because of caching). #2019-05-21 17:49:55 didibus In this case, I'm not interested in composing classpath dependencies. I'm not even coding. I want to install and run a Clojure app. #2019-05-21 17:50:29 didibus In that way, the alias is defining a package in the sense of Clojure app. #2019-05-21 17:53:24 didibus When tools.deps is used as a replacement for lein or as a build tool. Its kinds doing the same thing as well. In that you're adding all these convenient Clojure cli tools to your development environment. The difference is some of them also leverage tools.deps as the way to add your own code sources and resources which the build tool needs on the classpath as well. #2019-05-21 17:54:58 didibus Which is where composing classpath comes into play I guess. #2019-05-21 17:56:00 dominicm It would be really cool to have an upgrade bisect where an attempt is made to update your dependencies and then run the tests. If they still pass you're good, otherwise you remove the upgraded dependency which caused the tests to fail and try again #2019-05-21 17:56:24 didibus That's a great idea #2019-05-21 17:57:56 alexmiller This does not need to be in tools.deps #2019-05-21 17:59:02 didibus Ya, most likely is best as a tools for it. Could be added to one of the existing test-runners #2019-05-21 18:01:22 dominicm @didibus I was actually thinking that a tool which processed your deps.edn should take a "test" script to run that as long as it doesn't throw (or match whatever the clojure.test failure format is) #2019-05-21 18:02:34 didibus That's even better, that way its decoupled from the concrete test-runner and might be extended to work for all of them. #2019-05-21 18:24:10 dominicm Or even an arbitrary piece of code #2019-05-21 18:25:17 dominicm "This code used to return 10, which dependency changed that?" #2019-05-21 18:26:45 alexmiller would intersect well with some stuff Rich (in the form of codeq and REBL) and me (in the unreleased spec-aware testing tool) have worked on with respect to fine-grained code dependencies #2019-05-21 18:30:03 seancorfield ears perk up... "spec-aware testing tool"? Mmm... sounds lovely! #2019-05-21 18:30:35 alexmiller I've talked with you in the past about it, I'm pretty sure #2019-05-21 18:30:52 seancorfield It's been a while tho'... glad to hear work is still ongoing... #2019-05-21 18:31:27 alexmiller it's not. I mean it's not dead, but I haven't touched it in 2.5 yrs #2019-05-21 18:32:04 seancorfield haz a sad Hopefully it will eventually see the light of day? #2019-05-21 18:32:13 alexmiller but it was always part of the original spec idea and may eventually get revived #2019-05-21 18:32:57 alexmiller it may make more sense to build on top of codeq 2 whenever I get back around to it (which is also not released, but is in active development) #2019-05-21 18:33:25 alexmiller and starting to get real interesting when combined with some new things coming in REBL #2019-05-21 18:34:01 alexmiller and Datomic, and Clojure itself, now that I think of it #2019-05-21 18:34:09 seancorfield I'm always happy to see new features in REBL since it's a core part of my day-to-day workflow now. #2019-05-21 18:34:45 alexmiller next release will have some great new stuff in it, Rich has been in the hammock :) #2019-05-21 17:57:20 didibus Anyways, I'll think about all this some more and I'll stop with my thought dumping :drooling_face: #2019-05-21 18:25:04 alexmiller one of the goals with introducing clj was to encourage more "small" build programs. I think that's been pretty successful. I think it's also useful to step back with where we are now and look at how people are using those, what use cases/problems we could support better, and what we could do to help. Eventually, we'll get around to that. #2019-05-21 19:19:44 dominicm Running multiple programs seems to be the difficulty people keep stumbling over, from my observations at least. #2019-05-21 19:21:44 alexmiller like? #2019-05-21 19:22:45 alexmiller like a static set of things, or something where there is same input to all of them? #2019-05-21 19:27:59 dominicm Figwheel, and then nrepl are the two I mostly see. And then combining them with rebel.
It's not well suited to multiple mains because they need to background (except for rebel). But I also consider most of them individual choices rather than project ones. #2019-05-21 19:31:37 seancorfield That's why we wrote our build script: it takes a series of arguments and runs clojure for each one (plus some alias manipulation). #2019-05-21 19:32:38 seancorfield But the multi-main in one JVM is another use case that could be nice as a built in. It's no big deal to write your own -main to start up multiple things tho'... #2019-05-21 19:33:17 seancorfield We have a dev.repl/-main that starts up various combinations of tooling for that. #2019-05-21 19:45:38 alexmiller what if clojure.main took multiple -m's ? #2019-05-21 19:51:19 dominicm @alexmiller it might work, I've done a little thinking, and I'm guessing that most tools would need to take a -d flag.
The other side of this is "activation", the interface needs to be supportive to a user combining their own development programs with those of the project. #2019-05-21 19:51:58 alexmiller what's -d? #2019-05-21 19:52:17 dominicm Demonize. Sorry. #2019-05-21 19:52:41 alexmiller by "multiple -m"'s, I assuming each would have their own main + args #2019-05-21 19:52:54 alexmiller -m foo 1 2 -m bar 3 4 #2019-05-21 19:53:10 alexmiller so if foo or bar took -d, that would be fine #2019-05-21 19:53:14 dominicm Me too. I just mean that eg nrepl would need a flag to indicate it should background itself. #2019-05-21 19:53:41 alexmiller ah, unless they launched in parallel in different threads :) #2019-05-21 19:54:58 alexmiller definitely things to work through, but seems non-crazy to me #2019-05-21 20:13:46 seancorfield It would definitely be interesting if multiple -m options launched the first N-1 in threads and the last one ran directly -- and then it waited on all those threads to complete. #2019-05-21 20:18:45 ghadi no thanks #2019-05-21 20:19:03 ghadi sounds like that can be done in userspace #2019-05-21 20:19:41 ghadi if there were multiple mains, I'd expect the -main functions to return before moving on to the next one #2019-05-21 20:27:15 seancorfield Both options have their pros and cons. Maybe a -d option to run -main in a thread and -m to run -main sequentially is a good approach? #2019-05-21 20:20:15 ghadi there are so many choices in this neighborhood #2019-05-21 20:20:50 alexmiller I wouldn't leap to that conclusion #2019-05-21 20:21:18 ghadi the conclusion i deleted? simple_smile #2019-05-21 20:22:24 WhittlesJr Sorry to interrupt... is there a feature similar to lein clean in Clojure CLI? #2019-05-21 20:25:44 seancorfield clj -Sforce is probably the closest. #2019-05-21 20:26:17 seancorfield It ignores the cache, recalculates the classpath etc, and updates the cached values from that. #2019-05-21 20:26:46 WhittlesJr Hmm, ok. Thanks! #2019-05-21 20:28:52 seancorfield Although I guess you can get -d already with -e "(future ((requiring-resolve 'some.ns/-main)))" ... except for argument processing #2019-05-21 20:37:44 hiredman of course then you have to decide who gets to call shutdown-agents #2019-05-21 20:39:01 dominicm I guess the thread needs to have interruption configured for shutdown #2019-05-21 20:39:22 dominicm Or basically daemonization #2019-05-21 20:41:09 alexmiller let's just reinvent pods! :) #2019-05-21 20:41:25 seancorfield Or not... 🙂 #2019-05-21 20:45:09 dominicm Boot is coming for clj I believe, so maybe that will happen. #2019-05-22 00:49:08 seancorfield @hlolli That's the easiest (and only supported) way for tooling to get the list of system, user, and local deps.edn files. #2019-05-22 00:49:56 seancorfield On Windows, you'll either need to use the alpha PowerShell installer for the Clojure tools, or stick with WSL and one of the "bash on Windows" Linux options. #2019-05-22 00:50:35 hlolli yes, easy to fix, but this makes releasing libraries with lein+tools.deps a no-go #2019-05-22 00:50:54 seancorfield I'd recommend not using tools.deps with other build systems (as the author of boot-tools-deps) and just go with clojure / clj directly instead. #2019-05-22 00:50:55 hlolli I wonder if the function for describe wasn't originally written in clojure? #2019-05-22 00:51:19 hlolli yes I guess so, like I said, late to realize that limitation 🙂 #2019-05-22 00:51:40 seancorfield In a very early version of tools.deps it did not shell out to clojure. It was changed to do that to get around a lot of other problems people were having with tooling. #2019-05-22 00:53:03 hlolli well, tools.deps is still alpha, so let's see what happens. I'd root for keeping it as a library, no matter how much sweat and blood is needed (none of it coming from me 🙂 ). Then it also enables calling it straight from java without any deps. #2019-05-22 00:56:01 seancorfield At this point, it's pretty fixed in stone that it's going to shell out to clojure I think. #2019-05-22 00:56:23 seancorfield That's why there's been so much work done on getting a clojure command working on Windows (via PowerShell). #2019-05-22 00:58:48 seancorfield I initially wrote boot-tools-deps thinking that combining Boot with clj/`deps.edn` was a good approach but once I started using it, I realized it really wasn't. Boot's ideas of paths and dependencies don't mesh well with the Clojure tools -- and it was just easier to stop using Boot altogether. I would imagine the same is really true of Leiningen as well. #2019-05-22 00:59:48 hlolli yes, I'm starting to think that. Or I start writing tool dependent code if lein x elseif tools.deps y. Which I've never done with clojure. #2019-05-22 01:00:38 seancorfield These days I'm just targeting clj/`deps.edn` but I'm also releasing libs to Clojars so they can be used by lein/`boot`. #2019-05-22 01:01:11 seancorfield We switched our entire stack from Boot to clj/`deps.edn` last year. We've been very happy with that. #2019-05-22 01:01:53 hlolli yes I bet, it's just so unneeded I think, to tell everybody to change build tool, even tough tools.deps rocks and I would want everybody to use it #2019-05-22 01:03:23 seancorfield If folks are happy with lein/`boot`, stay with lein/`boot`. Don't try to blend them tho' 🙂 #2019-05-22 01:11:51 hlolli yes, I agree with the fact that the my world would be a better place if everybody used tools.deps #2019-05-22 01:12:07 alexmiller -Sdescribe doesn't need to shell out if there isn't an install deps.edn, which may happen #2019-05-22 01:12:52 hlolli ah that explains! the mix here of project.clj and deps.edn in a directory is probably confusing #2019-05-22 01:33:41 seancorfield @alexmiller Tooling that calls clojure-env depends on tools.deps shelling out -- to run clojure -Sdescribe. #2019-05-22 01:33:56 seancorfield Are you saying that tooling could figure out it doesn't need to call clojure-env? #2019-05-22 01:35:16 seancorfield @hlolli Which project are you looking at that has both project.clj and deps.edn? That usually means it's a Leiningen project that has added deps.edn so folks can rely on that project on GitHub directly from their own deps.edn... #2019-05-22 01:35:38 seancorfield (it doesn't mean the project is relying on Leiningen using tools.deps) #2019-05-22 01:35:41 alexmiller well what are you calling clojure-env for? #2019-05-22 01:35:52 alexmiller I assume the locations of the deps.edn files #2019-05-22 01:36:10 seancorfield Right, so that those can be passed back in to tools.deps... #2019-05-22 01:36:28 alexmiller there are 3 - install (goes away), ~/.clojure (deterministic based on env vars), current directory #2019-05-22 01:36:31 alexmiller the first one is the hard one #2019-05-22 01:37:29 seancorfield Ah, and you're going to fold the system/install one back into the library, right? #2019-05-22 01:37:34 alexmiller yes #2019-05-22 01:37:56 alexmiller this also removes the biggest pain in the ass in the installer which is also finding that path #2019-05-22 01:38:04 alexmiller well, maybe it's the path to the jar itself #2019-05-22 01:38:04 seancorfield (which is what I had originally done with boot-tools-deps -- baked in a copy of it from the Clojure tools repo 🙂 ) #2019-05-22 01:38:25 alexmiller yeah. doesn't even need to be a file, could just hardcode parts of it #2019-05-22 01:38:27 hlolli this project in question is Overtone, I wanted to add support for tools.deps, so I had to add badegion to extract the native deps. Badegion is what's calling tools.deps.alpha namespace. But in discssion with the maintainer of badegion, it seems I can do all that without deps.edn being in the jarfile. #2019-05-22 01:39:15 alexmiller the root deps.edn is just: #2019-05-22 01:39:18 alexmiller {
:paths ["src"]
:deps {
org.clojure/clojure {:mvn/version "1.10.0"}
}
:aliases {
:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.6.496"}}}
:test {:extra-paths ["test"]}
}
:mvn/repos {
"central" {:url ""}
"clojars" {:url ""}
}
} #2019-05-22 01:39:42 seancorfield OK, once tools.deps bakes that in and all the tooling has rev'd to depend on that version and everyone has updated to the latest versions of those tools, we will all be set! 🙂 #2019-05-22 01:41:11 seancorfield I am now remembering this discussion -- but it was quite a while ago and I guess I'd forgotten that it was seriously being considered. Good to hear @alexmiller #2019-05-22 01:41:38 alexmiller you can also just use tools.deps directly, hard-code your own deps.edn root with the stuff you need from that, and not ever call clojure-env #2019-05-22 01:42:18 alexmiller which is admittedly more work, but is also something that does not depend on anyone else doing anything #2019-05-22 01:42:18 seancorfield Yeah, but I think tooling has all evolved to assume that calling clojure-env is the "correct, supported way" to get that information :rolling_on_the_floor_laughing: #2019-05-22 01:42:24 alexmiller bleh #2019-05-22 01:42:30 alexmiller evil #2019-05-22 01:42:42 alexmiller I'll just change it to return "thppppt" #2019-05-22 01:46:04 hlolli @seancorfield amazing, thanks, my error is actually coming from right there but not tools.deps.alpha 😄 #2019-05-22 01:46:14 hlolli the windows cmd is hard to read #2019-05-22 01:46:34 alexmiller well I can't fix that :) #2019-05-22 01:46:44 hlolli exactly 🙂 #2019-05-22 01:46:59 seancorfield It had been a while since I last looked at the source of Rick's project... At least boot-tools-deps is archived now so folks will be discouraged from using it 🙂 #2019-05-22 15:07:22 WhittlesJr For a maven repo with a self-signed SSL cert, lein had me add the path to my system's ca-bundle.crt in profiles.clj / :user / :certificates. What do I do with tools.deps to make that work? #2019-05-22 15:10:46 WhittlesJr Up until now I've been able to forget about our internal maven server and just use :git/url, but there's a java project out there (https://github.com/entropia/libsocket-can-java) that never released to maven central that we had to build and host ourselves. #2019-05-22 15:30:19 alexmiller nothing in tools.deps that is similar. what happens if you try it? #2019-05-22 16:09:47 WhittlesJr error in process sentinel: nrepl-server-sentinel: Could not start nREPL server: Error building classpath. Failed to read artifact descriptor for de.entropia:libsocket-can-java:jar:0.3.0
...
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
It might be my OS configuration, I'm looking into it. #2019-05-22 16:31:57 alexmiller usually you would update cacert in your Java installation for this I think #2019-05-22 16:35:04 alexmiller that's probably enough to google for how to do it #2019-05-22 20:07:42 borkdude I’m trying to get rid of brew in a Mac environment and trying the install: illegal option -- D
Any workarounds for this not involving brew? #2019-05-22 20:09:06 borkdude the reason is that brew from scratch is terribly slow on CI #2019-05-22 20:11:37 ghadi you can cp + chmod instead of install #2019-05-22 20:12:01 ghadi you can mimic the "spirit" of the install script #2019-05-22 20:12:20 borkdude yeah, that works 🙂 #2019-05-22 20:14:21 alexmiller you can also just look at what the brew script does (which is largely to put the tar in a place and run its own variant of the install script) #2019-05-22 20:14:43 borkdude that works too #2019-05-22 20:16:24 borkdude that sounds easy enough, ok #2019-05-22 20:17:15 alexmiller I guess it matters where it downloads and does that stuff and that's the env brew provides #2019-05-22 20:18:27 alexmiller ultimately there is clj/clojure, a jar file, deps.edn file, etc and it's just a matter of putting those all in the right places and pointing at each other #2019-05-22 21:43:07 borkdude got it working now #2019-05-22 21:49:23 borkdude clojure now installs in 0 seconds in CI instead of 5 minutes 😉 #2019-05-22 23:09:17 hlolli Is there a way to set warn-on-reflection true globally in tools.deps, some trick that someone has found out? #2019-05-22 23:11:47 hlolli nevermind, I should get used to just keeping it in my clojure files #2019-05-22 23:19:40 alexmiller I think there’s a system property for that? Not at a computer to look it up. #2019-05-22 23:21:06 hlolli off-topic, I'm in the process of trying to get standalone jar to run, is it safe to ignore loop/recur symbol warnings that end with Auto-boxing loop arg: some-symbol ? #2019-05-22 23:25:49 alexmiller Depends whether you care about perf #2019-05-22 23:31:53 hlolli ok, so it's possible to remove these reflections, my misunderstanding #2019-05-22 23:32:02 hlolli this looks about right? :jvm-opts ["-Dclojure.compile.warn-on-reflection=true"] #2019-05-22 23:33:15 alexmiller Looks good as far as I remember off the top of my head #2019-05-22 23:34:10 alexmiller Oh yeah, so then no :) #2019-05-22 23:34:56 hlolli I'm running compile/compile, and I only hear crickets #2019-05-23 00:17:23 seancorfield @hlolli, for now, you have to have (set! *warn-on-reflection* true) in each source file. I think it's a good habit to get into. #2019-05-23 02:28:30 alexmiller you can specify :deps/root as a subdir in which to find deps.edn/pom.xml in the context of a git lib #2019-05-23 02:29:17 alexmiller doesn't look like that project needs that though? there's a deps example in the readme #2019-05-23 08:03:30 ewen Is there a way to add clojure$slim (slim classifier) as a dependency and exclude the clojure dependency that comes with tools.deps ? Adding clojure$slim to deps.edn results in both clojure and clojure$slim to be on the classpath. Looks like a consequence of the recent classifier change ? #2019-05-23 08:30:06 dominicm Ah, because it comes in from the "system" deps.edn, that's interesting. #2019-05-23 08:30:58 dominicm I guess the original :classifier design was better for this case. #2019-05-23 11:16:06 alexmiller no not possible right now. Classifier deps really are treated as separate deps and because Clojure is included by default, there is no way to remove it #2019-05-23 11:18:26 alexmiller Hadn’t ever thought of this re slim (almost no one uses it) #2019-05-23 14:44:15 lilactown is there a way to determine a proper “groupId” from a deps.edn? not really right? #2019-05-23 15:24:11 alexmiller You mean for deps or for the project itself? #2019-05-23 15:29:55 lilactown for the project itself #2019-05-23 15:32:21 alexmiller then, no #2019-05-23 15:32:41 lilactown not super important, I’m just writing a utility to add a dep to a project via command line. ran into where:
- plum add org.clojure/clojure "1.10.0" appends org.clojure/clojure {:mvn/version "1.10.0"}
- plum add "some-long-sha" right now adds as
clojure/clojure {:git/url "" :sha "some-long-sha"}
#2019-05-23 15:33:20 lilactown the pragmatic thing to do would be to check for a pom.xml in the repo, I suppose. or just not worry about it #2019-05-23 15:34:35 alexmiller well using clojure as a git dep like that won't work (as it's half java and must be compiled into an artifact) #2019-05-23 15:35:28 lilactown ah, fair point #2019-05-23 15:36:51 lilactown I think the problem exists still as: I want to use the latest version on git of a library that I previously was consuming through maven. with my naive approach, a user might end up with two different identifiers in their deps.edn for the same lib #2019-05-23 15:37:02 lilactown which, btw, what would the behavior be in that case? last one wins? #2019-05-23 15:39:00 alexmiller right now, it will error I think #2019-05-23 15:39:35 alexmiller it is, however, actually possible to resolve as maven artifacts include their git commit info at build time, making it possible to resolve the "newer" one via git commit comparison #2019-05-23 15:39:56 alexmiller I have not actually done that work yet, but have done enough research to believe it's feasible #2019-05-23 16:57:54 lilactown ah neat. thanks for the info! #2019-05-23 17:51:54 WhittlesJr Yesterday I was asking about adding a self-signed SSL cert with tools.deps, and I did some research on it. I'm on NixOS, which is a functional / immutable OS, and as such its Java trust store is also immutable (read-only), so I can't just imperatively add the cert to the trust store using keytool. Unfortunately right now there's not a configuration option in NixOS to do this either. Am I SOL?
https://stackoverflow.com/a/49423656/2015090 #2019-05-24 00:21:51 mynomoto I generated a pom for my project using -Spom so I could use dependency-check-maven to find vulnerabilities on dependencies. But since the way maven and tools.deps solve dependencies I got discrepancies on the dependency tree calculated by maven and by clj. Could -Spom return all dependencies including indirect ones or have another command that could do that? #2019-05-24 00:34:16 alexmiller the trees are really different because mvn and clj resolve dependencies differently. which one do you actually care about? #2019-05-24 00:34:46 alexmiller you can grab the actual full set of deps clj is resolving (as data) by looking at the .libs file that's generated #2019-05-24 00:35:43 alexmiller if you do clj -Sverbose you'll see a cp file like .cpcache/723570742.cp - if you replace .cp with .libs, that's the file #2019-05-24 00:36:51 alexmiller that's an edn file, so you could suck it in with clojure and manipulate it however you need #2019-05-24 00:41:56 mynomoto I care about the clojure one, since that will go to production, but the tool to check dependencies uses the pom information. #2019-05-24 00:43:41 mynomoto So reading the .libs file and generating a pom.xml with all dependencies should work. #2019-05-24 01:18:08 seancorfield @alexmiller Does that mean that deps-based projects that use clj -Spom to generate/maintain a pom.xml that is published to Clojars can cause projects that use them to get the "wrong" dependencies? #2019-05-24 01:29:30 alexmiller the generated pom should have only the same top level deps #2019-05-24 01:30:48 alexmiller my understanding of the request above is that he is scanning the full transitive dep tree, which may indeed be different between clj and (mvn or lein) #2019-05-24 01:34:19 mynomoto This is accurate. #2019-05-24 01:31:51 alexmiller these differences are presumably mostly same libs but different versions, but in cases where transitive deps change, could be different deps #2019-05-24 01:33:46 alexmiller in theory, that may be an issue. in practice, it seems to rarely matter, perhaps because most libs are subsequently used as someone else's lib, and then lose all control over their dep versions anyways. #2019-05-24 01:40:45 seancorfield Thanks for the clarification! #2019-05-24 02:12:47 mynomoto I have mixed feelings about my experience with the classpath differences today. lein and boot do the same as Maven in resolving transitive dependencies so this is a tools.deps only problem. If -Spom is supposed to create a classpath for libraries it's doing fine with the caveat that if you want to analyze the tools.deps library classpath using the generated pom using Maven tooling you can get results that are not real, but as Alex pointed out, in the end is the application classpath that matters. In the end I think that having an easy way of generating a pom.xml that will create the same classpath as tools.deps is important because of tooling that already exists for Maven. #2019-05-24 02:20:35 mynomoto dependency-check-maven plugin pointed several high/critical vulnerabilities on dependencies used by pedestal and buddy mentioning only libraries that I think are popular in the clojure world. I had no time to check if those could be exploited. This is on the latest release of both libraries. The dependencies have new releases not affected by the vulnerabilities. #2019-05-24 02:30:29 alexmiller my limited experience with dependency-check-maven and the underlying stuff is that the matching is very bad and often highlights libraries that have nothing to do with the CVEs it associates with it #2019-05-24 02:33:31 alexmiller an example is https://github.com/jeremylong/DependencyCheck/issues/1788 which flagged data.zip afaict because it had "zip" in the name and that matched a totally unrelated "zip" project. I dug into the code just enough that I have a really low confidence in the results of this tool. So, fyi. #2019-05-24 02:34:05 mynomoto Having spent a fun afternoon reading cves, looked legit. Cves were very specific on coordinates and versions affected. #2019-05-24 02:35:17 alexmiller if you have a list of issues, would be great if you could just drop it somewhere so we could do necessary stuff to improve things #2019-05-24 02:35:56 alexmiller issues on specific tools would be better, but if you've at least got a list, that would make it faster for someone else to log if you don't have time #2019-05-24 02:37:58 alexmiller on the suggestion re generating a pom, the goal was to create a pom that was similar to the deps.edn, which meant just the top level deps. Stating all lower level deps in the pom would be (conceptually) much different. That's more than I'm planning to take on for -Spom (which itself may get pulled out of tools.deps/clj and put into a standalone tool) #2019-05-24 02:38:57 alexmiller making a tool that used tools.deps and a deps.edn and did the dependency checks directly without involving Maven would be even better #2019-05-28 01:27:36 Richard Gould We've written something internally that we've been using for a few months to do this. Have just gotten clearance to open source it, hope it's useful. Available now at https://github.com/ROKT/check-deps
@U9Y2R7YAV #2019-05-24 02:51:13 mynomoto I understand the trade-offs of generating a pom similar to what other tools generate. But that comes with the inconvenience of having different classpaths on different tools because the algorithm used for transitive dependencies is different and the results are different than one would expect. While a tools.deps specific dependency check tool would be indeed better that would mean another tool that someone would have to maintain. Using one that already exists and is maintained has lots of advantages too. I will rerun the analysis without my dependencies overrides and report on each tool tomorrow. #2019-05-24 02:57:32 alexmiller thanks, that would be great #2019-05-24 04:07:32 alexmiller looks like good reports, thx! #2019-05-24 09:33:27 wei is there a good way to share code between two tools.deps projects living in the same git repo? #2019-05-24 09:39:01 dominicm @wei use a local dependency when developing them, I think that's what you're asking. #2019-05-28 09:41:21 thheller @alexmiller I think a shadow-cljs user stumbled into a bug in tools.deps regarding :exclusions of libs. not sure if intentional. #2019-05-28 09:41:30 thheller {:deps
{org.clojure/clojure {:mvn/version "1.10.0"}
thheller/shadow-cljs {:mvn/version "2.8.37"}
io.pedestal/pedestal.service {:mvn/version "0.5.5"}
}} #2019-05-28 09:41:59 thheller shadow-cljs has a dependency on ring-core but excludes clj-time #2019-05-28 09:42:00 thheller [ring/ring-core "1.7.1"
:exclusions
;; used by cookie middleware which we don't use
[clj-time]] #2019-05-28 09:42:46 thheller pedestal.service also depends on ring-core but the exclusion from shadow-cljs is applied to its dep #2019-05-28 09:43:51 thheller so clj-time is missing #2019-05-28 10:19:53 rickmoynihan Do any of the tools-deps tools for generating an uberjar have support for something like lein’s :uberjar-merge-with? #2019-05-28 10:20:28 dominicm Pack avoids conflicts altogether with a classloader. #2019-05-28 10:21:53 rickmoynihan The thing is, I need to merge edn files #2019-05-28 10:22:41 rickmoynihan e.g. data_readers.edn but in particular duct_hierarchy.edn #2019-05-28 11:03:51 rickmoynihan ok actually avoiding the conflicts and using a classloader would probably work… I guess I can just walk the resources and merge them myself.
Which of the uberjar options will use a classloader? I’m guessing capsule? #2019-05-28 11:05:53 rickmoynihan hmmm… infact I probably don’t need to merge them myself… duct must be doing this already to work at the repl. #2019-05-28 11:17:23 rickmoynihan ok I can confirm pack.alpha capsule just works ™ thanks #2019-05-28 11:33:49 rickmoynihan also capsule is awesome 😀 #2019-05-28 12:54:48 dominicm Capsule is very cool :) #2019-05-28 12:55:52 dominicm Pack currently has a no conflict policy so that you don't have an option to shoot yourself in that respect. #2019-05-28 13:54:09 rickmoynihan 👍 a good policy to have #2019-05-28 13:56:02 rickmoynihan what is the purpose of -e in pack.alpha? Why not just an alias with the extra-paths there? Or is it different somehow? #2019-05-28 13:56:32 rickmoynihan Is there any extra meaning in
> Add directory to classpath #2019-05-28 14:23:39 dominicm @rickmoynihan no extra meaning. That flag will continue to exist so that I can put directories generated with mktemp into the class path. #2019-05-28 14:24:09 rickmoynihan ahh makes sense thanks #2019-05-28 14:25:07 dominicm The "for building" is probably a bias in my writing for my particular use case. #2019-05-28 14:27:03 rickmoynihan yeah — but that use case makes sense 🙂 the docs could possibly benefit from a clarification as to that purpose… but no biggie 🙂 #2019-05-28 14:27:53 rickmoynihan it is essentially the same as :extra-paths but more suited for scripting/api usage in wider build processes #2019-05-28 14:36:21 dominicm I'm terrible at writing documentation :) #2019-05-28 17:33:24 rickmoynihan @dominicm: incidentally it seems that pack.alpha doesn’t seem to honor :extra-paths in other aliases you supply during the pack phase. I’m guessing this is a bug? Though I could have my wiring wrong. #2019-05-28 17:33:50 dominicm Hmmm, that would be a bug, yeah. #2019-05-28 17:34:00 dominicm I thought I tested that though #2019-05-28 18:05:06 Daniel Hines Reposting noob cli question from #beginners:
Is there a way to create a “default profile” for the clj cli? Such that whenever I run clj, certain deps are always merged in? #2019-05-28 18:05:51 Daniel Hines like a ~/.deps.edn file or something? #2019-05-28 18:11:15 rickmoynihan you can include custom (private) aliases in ~/.clojure/deps.edn but as far as I know you always have to opt in to aliases #2019-05-28 18:11:30 rickmoynihan tools.deps prefers to be explicit #2019-05-28 18:12:35 didibus Yes, put a deps.edn file here $HOME/.clojure #2019-05-28 18:16:02 didibus Nah, you can just add normal deps #2019-05-28 18:16:07 didibus And they'll always be included #2019-05-28 18:16:39 didibus That's how Clojure itself gets loaded #2019-05-28 18:17:48 Daniel Hines Also, -S seems to be missing from that list of possible aliases. #2019-05-28 18:17:55 didibus There are 3 deps.edn. The install one, the one in your home directory. And optionally, your project one. The :deps map gets merged from install to home to project. In case of conflicts, the later ones override the prior ones. #2019-05-28 18:19:35 didibus Aliases are like dynamic modifications #2019-05-28 18:20:40 didibus Normally, the classpath, that is, the path to all libraries available to your program is defined by the :deps map, and the :paths vector, and a few others. #2019-05-28 18:21:08 didibus But, maybe on occasions, you want to start a program or a repl with more or less libraries included #2019-05-28 18:21:19 didibus That's what aliases let you do #2019-05-28 18:21:36 didibus So you can define sets of dependencies and name them with an alias. #2019-05-28 18:22:06 didibus Now, from the command line, you can choose to include some of those additional sets by adding the alias for it #2019-05-28 18:23:37 Daniel Hines That’s a very helpful explanation @didibus, thanks. So including aliases merges the options contained in those aliases? #2019-05-28 18:23:50 didibus Ya #2019-05-28 18:24:12 Daniel Hines You said it’s possible exclude as well? How would that be accomplished, since it’s a merge? #2019-05-28 18:24:18 didibus Pretty much. Now the way things are "merged" can vary depending in the type of alias and config. #2019-05-28 18:25:45 didibus Well, you can't exactly exclude. So you'd be better off thinking of them more as inclusions #2019-05-28 18:26:05 didibus But for example, you can specify :override-deps which will override versions #2019-05-28 18:26:50 didibus Or :classpath-overrides which overrides the path to a lib #2019-05-28 18:29:05 seancorfield That has some usage notes as comments as well. Plus the README in that repo explains in more detail... #2019-05-28 19:25:10 Daniel Hines This is super helpful, thanks! #2019-05-28 18:33:31 didibus I'm not sure if you could classpath-override to a null dir #2019-05-28 18:33:42 didibus Or a dir which doesn't have the dependency #2019-05-28 18:33:48 didibus As a weird way to exclude #2019-05-28 18:33:59 didibus But I wouldn't recommend it #2019-05-28 18:34:23 didibus Better design it so aliases are additions or overrides, but not removals #2019-05-28 18:39:07 didibus Also, I'm not sure what you mean by -S, that's just a prefix for other cli options. It's not related to aliases. #2019-05-29 17:55:14 rickmoynihan when using :git/deps does :deps/root need to specify a directory containing a deps.edn file or can it be a path to a deps.edn file named something arbitrary, like foo.edn? #2019-05-29 17:55:54 rickmoynihan I’m guessing it must be a directory #2019-05-29 18:08:36 dominicm Yeah, directory #2019-05-29 18:28:48 rickmoynihan I’m assuming it’s not possible to override just the :deps/root for a :git/dep in aliases? Use case is sharing a single :git/url and :sha across multiple aliases that parameterise just the :deps/root.
I’m guessing that to do this I would need to duplicate the :sha and :git/url as :extra-deps in each alias. #2019-05-29 19:15:18 dominicm Yeah, duplication #2019-05-29 19:15:28 dominicm I'd write a tool to manage it for me #2019-05-29 19:35:01 holyjak Is there a trick to not need to repeat the version number here: {:deps {org.apache.lucene/lucene-core {:mvn/version "8.1.1"}
org.apache.lucene/lucene-queryparser {:mvn/version "8.1.1"}
org.apache.lucene/lucene-analyzers-common {:mvn/version "8.1.1"}} ? #2019-05-29 19:47:13 seancorfield Nope. #2019-05-29 19:47:41 seancorfield Unless there's a bundled artifact you could depend on that would bring in all those pieces. #2019-05-30 09:42:48 dm3 How do I specify repository mirrors with deps.edn? #2019-05-30 09:49:55 rickmoynihan How are people deploying artifacts to private s3 buckets with tools.deps? Are people using mvn deploy, or scripting up pomegranate aether with s3-wagon-private somehow? #2019-05-30 09:56:35 rickmoynihan pomegrante’s aether deploy looks like it might be easy enough… though not sure how to wire in the s3 wagon… #2019-05-30 09:57:07 rickmoynihan It looks like it might be automagically done #2019-05-30 10:32:11 rickmoynihan ok it looks like it’s pretty easy to get pomegranate deploy with s3 wagon working in a simple clj script #2019-05-30 11:00:44 dominicm @rickmoynihan write that down somewhere for others! #2019-05-30 11:04:12 rickmoynihan Will do. It looks like it basically just works. I’m sure it’d be easily extractable into an aliasable task, though I haven’t thought too much about how it should work in that form. #2019-05-30 11:05:51 rickmoynihan Right now I essentially want to programmatically generate a bunch of very similar artifacts that differ only in terms of one or two files and their artifact id. #2019-05-30 11:08:14 rickmoynihan The higher level problem I’m looking to solve is we have a repo that contains our own css framework, and we want to package client themes from a single place but consume them downstream as essentially webjars. So each clients theme will typically differ only by their theme.css #2019-05-30 11:09:22 rickmoynihan so I’ll likely solve that problem first, then look at extracting something reusable. #2019-05-30 19:55:58 mnewt Is there a way to specify a dependency for a leiningen project? (i.e. project.clj is the only manifest) #2019-05-30 20:49:54 mnewt @ballpark Thanks for that. That is useful but unfortunately not quite what I’m looking for. I was hoping that there was a way to pull in an existing leiningen project without modifying it. Is there a library or something that can resolve dependencies from a project.clj file or is the only way to do that to run leiningen itself? My use case is I would like to include specific versions of projects directly from github. #2019-05-30 21:10:10 seancorfield @mnewt Petition the maintainer to add deps.edn 🙂 You could always override the deduction, specify the manifest as :deps and then provide the transitive dependencies yourself in your deps.edn file. #2019-05-30 21:12:48 mnewt @seancorfield Thanks, that makes sense #2019-05-31 12:48:50 dharrigan If I have a configuration in my $HOME/.clojure/deps.edn file, thus: {:extra-deps: {foo-bar {:local/root "libs/foo}}}, it doesn't appear that resolution happens relative to the $HOME/.clojure" directory, I have to put the full path in, i.e., ....{:local/root "/home/foo/.clojure/libs/foo" #2019-05-31 12:50:01 dharrigan Is my thinking correct on this? No way for the resolution to happen relative to where the deps.edn file is? #2019-06-03 12:41:31 rickmoynihan I think that’s correct, as things will get more complex when you have transitive git deps etc. #2019-06-03 12:54:53 dharrigan thanks for the confirmation 🙂 #2019-05-31 22:15:37 sparkofreason I still have this exception running clojure from WSL. I've confirmed that ssh-agent is running and configured. com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available: ssh-agent:jna,ssh-agent:nc Does anybody know the solution? #2019-05-31 23:06:19 hiredman You need some native code to talk over Unix sockects on the jvm, which is how you interact with ssh-agent. The jsch library might not be able figure out the native code required for wsl #2019-06-01 00:28:10 sparkofreason I assume people have this working, and there's just something about my particular WSL install (pengwin) that is an issue. #2019-06-01 02:06:04 Travis Does anyone know if there is a way to override where tools-deps looks for the .m2 directory ? For instance with Maven you can do something like
MAVEN_OPTS=-Dmaven.repo.local=/cache/.m2
#2019-06-01 11:01:03 alexmiller You can set :mvn/local-repo at the root in deps.edn #2019-06-02 22:17:08 Travis @U064X3EF3 is there a way to set this at the command line for override in some build env? #2019-06-02 22:22:29 alexmiller You can use -Sdeps with an in-line deps.edn to add it on the command line #2019-06-01 11:08:13 pez Hello. I am trying to get the Calva jack-in story to be told nicely on Windows. Can I assume that powershell is used for cli/deps? #2019-06-01 11:09:50 pez Also I am a bit confused about when the command line tool is named clj and when it is named clojure. #2019-06-01 12:19:41 sogaiu @pez in *nix and macos isn't one just a wrapper to use the other along with rlwrap? have a look inside the files? #2019-06-01 12:23:59 pez Yes, checked the contents on my Mac now. clj wraps clojure in rlwrap. So it is clojure I should use for jack-in then. Thanks! #2019-06-01 17:49:48 yuhan but I could not figure out how to get from that to a clj / clojure executable on the PATH #2019-06-03 17:11:05 rickmoynihan I’m running a clojure CLI tool inside travis, and one of the deps is in a private github repo; and it dies with an error like:
Error building classpath.
The build itself is itself inside a private repo too, and travis seems to be able to clone that ok.
Any ideas what I might need to do? #2019-06-03 17:16:33 dominicm @rickmoynihan I would guess ssh-add or similar #2019-06-03 17:16:42 rickmoynihan yeah #2019-06-03 17:16:59 dominicm Is an agent running? #2019-06-03 17:17:22 rickmoynihan I would’ve expected travis to already have that setup though, given that one of the first things it does is clone a private GH repo #2019-06-03 17:17:52 dominicm Yeah. I wouldn't be certain it's using an agent #2019-06-03 17:17:59 rickmoynihan fair #2019-06-03 17:18:06 dominicm The other thing is, are you using gitlab? #2019-06-03 17:18:35 rickmoynihan github #2019-06-03 17:36:18 dominicm Okay, then the other bug I'm thinking of doesn't apply #2019-06-03 17:48:52 alexmiller clj uses jgit/jsch (not git) so it's definitely possible to have git work on the initial clone and then tools.deps fail to clone a dep repo #2019-06-03 17:50:59 alexmiller there are a number of deficiencies in the jgit/jsch stuff (we're on a somewhat old version now too). definitely needs some attention and we're considering either using a newer version or calling out to git or possibly making both available #2019-06-03 20:16:42 WhittlesJr Does tools.deps support loading native .so libs via JNI? With leiningen, I had:
:resource-paths ["lib"]
:native-path "%s/lib"
:jvm-opts [~(str "-Djava.library.path=lib/:" (System/getenv "$LD_LIBRARY_PATH"))]
And that did the trick. I've tried a few different things but can't get this working with tools.deps (I get an UnsatisfiedLinkError.) #2019-06-03 20:17:40 alexmiller no, this is not yet supported #2019-06-03 20:18:25 alexmiller tools.deps can add paths to the classpath and use jvm opts, but it doesn't have support for unpacking or loading native libs #2019-06-03 20:19:01 WhittlesJr Dang, alright. Thanks. #2019-06-03 20:29:15 dominicm @alex.joseph.whitt what lib is this? Most of the ones I've seen unpack themselves. #2019-06-03 20:33:19 alexmiller lein has a thing I think that unpacks native libs #2019-06-03 20:33:23 alexmiller I don't know the details #2019-06-03 20:37:13 WhittlesJr It would be real nice if this was just built in #2019-06-03 22:01:23 jeroenvandijk @rickmoynihan Almost bedtime, but you can add a user's git keys with circleci and this allows you to use private libs (assuming they are accessible with those keys). I can have a look tomorrow if necessary #2019-06-04 09:55:14 martinklepsch curl -O
clj -Sdeps '{:deps {org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu #:local{:root "mxnet-full_2.11-linux-x86_64-cpu-1.4.1.jar"}}}'
Error building classpath. 1 problem was encountered while building the effective model for org.apache.mxnet:mxnet-core:[unknown-version]
[FATAL] Non-resolvable parent POM for org.apache.mxnet:mxnet-core:[unknown-version]: Could not find artifact org.apache.mxnet:mxnet-parent:pom:INTERNAL in central () @
Is this supported and a bug or am I doing something wrong here? #2019-06-04 11:17:27 alexmiller You might need to add a maven repo if that parent pom isn’t in maven central #2019-06-04 14:51:33 martinklepsch The weird thing is that this works:
clj -Sdeps '{:deps {org.apache.mxnet/mxnet-full_2.11-linux-x86_64-cpu {:mvn/version "1.4.1"}}}'
#2019-06-04 14:53:50 alexmiller we do have a known issue that repos defined in the pom don't get used and that may be the issue here #2019-06-04 14:54:26 martinklepsch but then the above should also not work, right? #2019-06-04 14:56:20 alexmiller when you load a jar dep, tools.deps is reading the model and resolving the deps #2019-06-04 14:56:48 alexmiller I think it's in the tools.deps pom reader where more needs to be done #2019-06-10 09:11:02 martinklepsch I'm not sure if TDEPS-46 s whats going on here. Just let me know if you want a separate JIRA to document this failure case #2019-06-10 12:40:47 alexmiller you could leave a comment on that ticket so it can be checked and spun out if needed #2019-06-04 11:30:35 dm3 are repository mirrors supported? #2019-06-04 12:13:51 alexmiller I don't think so? I think that's still a todo #2019-06-04 15:38:16 WhittlesJr @dm3 I've got repository mirroring working, if we're talking about the same thing. My deps.edn config includes:
:mvn/repos {"central" {:url ""}}
And my ~/.m2/settings.xml:
https://gist.github.com/WhittlesJr/3422e3374c0393188c1321bbd7605bd5
We're using Nexus 3 for our maven server ("my-mirror"). Caches everything and works like a charm. #2019-06-04 15:42:11 WhittlesJr (I had to use http because I couldn't add a self-signed certificate to the java trust store in my particular case) #2019-06-04 15:44:02 WhittlesJr I suppose that having all that config in settings.xml breaks repeatability though, so a pure tools.deps solution would be very nice indeed. #2019-06-04 15:45:46 WhittlesJr (Er, maybe it is repeatable after it's all cached? Perhaps you could drop the settings.xml after that point and it would still download what it needed from "my-mirror." I'm not sure.) #2019-06-04 15:51:44 alexmiller well that's not mirroring, that's just replacing the repo #2019-06-04 15:51:50 alexmiller that should work #2019-06-04 15:52:17 alexmiller maven actually has support for mirrors (or maybe the term is different, I can't remember) #2019-06-04 15:52:39 alexmiller mirrors being duplicate repos that can serve the same content #2019-06-04 16:02:27 WhittlesJr Yeah I think I've seen the word "mirror" used for both meanings so I'm not sure what term to use. It is at least called "mirrors" in settings.xml (the <mirrors> section). On first request, Nexus will download and cache the needed artifacts from maven central or clojars, and then the client downloads everything from Nexus. Is that different from what you're talking about? #2019-06-04 16:03:40 WhittlesJr I think I've also heard it called "proxying" #2019-06-04 16:05:14 WhittlesJr But that's also confusing because "proxying" usually refers to using an http proxy #2019-06-04 19:06:16 dm3 yeah, you’re overriding the repo #2019-06-04 19:06:32 dm3 mirror is another layer #2019-06-05 13:40:19 sveri @alexmiller Hey, I just came back from a longer break and noticed tools-deps is finally available on windows, which is pretty awesome, thanks 🙂 #2019-06-05 13:50:54 alexmiller give it a whirl #2019-06-05 13:55:05 sveri Yea, I just did and after I allowed myself to run unsigned scripts in the powershell installation worked and I can run it #2019-06-05 13:55:10 sveri Just about to experiment with it finally #2019-06-05 17:44:20 markx Hi all, why is sha required for a :git/url? Is it possible to automatically use master branch? #2019-06-05 17:45:57 hiredman you can't cache a reference #2019-06-05 17:46:41 hiredman clj startup performance is based building a classpath once and caching it for the same deps.edn file #2019-06-05 17:50:27 markx Ah makes sense. It doesn’t have a record of the local version.
I was just hoping to achieve some function like npm or pip or other package mangers. #2019-06-05 17:56:32 markx I still need to wire my brain to connect the repl and the cli tool. I understand the code but don’t know how to do it on cli. #2019-06-05 17:58:00 markx What I really really want is something like clj install xxx then I can just run xxx from my shell. #2019-06-05 17:58:55 markx What I really don’t like is to build or download a jar and manually put it into PATH. #2019-06-05 17:59:12 lilactown hey markx. a thing I'm building might help #2019-06-05 17:59:18 lilactown is this for installing libraries, or installing applications? #2019-06-05 17:59:40 markx Ideally for both, but mostly for applications. #2019-06-05 18:03:47 lilactown it doesn't have direct support for installing global aliases right now. but it does have a plum add command that adds a dependency to a local deps.edn #2019-06-05 18:12:32 dominicm @U4YGF4NGM do you have a plan to preserve whitespace when you do this? #2019-06-05 18:13:16 lilactown it should preserve whitespace #2019-06-05 18:13:32 lilactown well, it formats it using cljfmt #2019-06-05 18:14:19 lilactown which I suppose is not the same 😅 #2019-06-05 18:14:34 dominicm I would not use this, because that will lose comments. #2019-06-05 18:14:54 lilactown no, it will preserve comments. that is why I moved to using cljfmt / rewrite-clj in the first place #2019-06-05 18:15:06 lilactown go ahead and give it a shot #2019-06-05 18:15:10 dominicm Oh, you're using rewrite-clj directly too :) #2019-06-05 18:15:25 dominicm Good. That's what I was going to suggest if you didn't have an idea. #2019-06-05 18:15:40 dominicm But I was hoping you'd found a novel approach I didn't think of :) #2019-06-05 18:15:47 dominicm I'll look through the code to compare. #2019-06-05 18:17:30 dominicm I had some users get quite annoyed that they lost their carefully aligned arguments when I first added this, just fyi #2019-06-05 18:18:00 lilactown that makes sense. I wish there was a way to do preserving modifications of EDN better #2019-06-05 18:19:12 dominicm I spent some time on it in ednup #2019-06-05 18:19:30 dominicm The goal was to make a clojure map you could assoc without losing formatting #2019-06-05 18:20:21 lilactown I wonder if we could format the modification and then assoc that using rewrite-clj to not mess with the rest of the file? #2019-06-05 18:57:42 dominicm What I did in ednup was look at the previous key, and match the whitespace based on that #2019-06-05 18:11:21 markx This is a nice tool. I read about it on clojureverse.
IIUC, it provides cli commands to avoid manually editing the deps.edn file. But it doesn’t solve all my pains. I hope it can go one step further.
Do you plan to support installing global aliases? #2019-06-05 18:12:51 lilactown it could be added. if it would help you, create an issue on github. #2019-06-05 18:13:02 lilactown I would also look over a PR if you're interested in contributing 😄 #2019-06-05 18:40:43 markx Yeah I’m interested. I’ll try. But don’t wait for me. 😛 #2019-06-06 07:07:25 sveri How do I start a repl in a given namespace? I have not seen anything in the documentation about it. #2019-06-06 08:21:57 sveri I figured I could use something like this:
:aliases
{:cursive {:main-opts ["-e" "(load \"de/sveri/getless/user\")"
"-e" "(de.sveri.getless.user/reset)"]}}
It works, but is it the right way to use tools-deps? #2019-06-06 12:10:46 alexmiller You would need to do something like this #2019-06-06 12:11:30 alexmiller You could simplify to one expr by using requiring-resolve (added in 1.10) #2019-06-06 12:13:27 alexmiller ((requiring-resolve ‘de.sveri.getless.user/reset)) #2019-06-06 12:32:49 sveri @alexmiller Thanks, that works and looks better #2019-06-06 12:41:52 alexmiller although double parens always looks weird #2019-06-06 22:38:20 holyjak Is :local/root supposed to work for pointing to a local lein project (with a pom.xml)? Or only deps projects? Thanks! (It doesn't for me) #2019-06-06 22:58:37 alexmiller it should work for deps.edn and pom.xml #2019-06-06 22:58:48 alexmiller in what way does it not work? #2019-06-07 09:44:25 holyjak E.g. REPL does not see the code from that project: $ clj -A:test
Clojure 1.10.1-beta2
user=> (require '[clj-concordion.core :as cc])
Execution error (FileNotFoundException) at user/eval1 (REPL:1).
Could not locate clj_concordion/core__init.class, clj_concordion/core.clj or clj_concordion/core.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.
with deps.edn including {:aliases
{:test
{:extra-deps {clj-concordion {:local/root "/Users/me/Projects/clj-concordion"}
;clj-concordion {:local/root "/Users/me/Projects/clj-concordion/target/clj-concordion-0.0.5.jar"} (the .jar works) and $ head ~/Projects/clj-concordion/pom.xml
<?xml ...><project ...>
<modelVersion>4.0.0</modelVersion>
<groupId>clj-concordion</groupId>
<artifactId>clj-concordion</artifactId>
...
<build>
<sourceDirectory>src</sourceDirectory>
...
and $ head -n1 ~/Projects/clj-concordion/src/clj_concordion/core.clj
(ns clj-concordion.core
(Could there be a problem with using src/ instead of src/main/clojure/?) #2019-06-07 16:05:49 mss is there a function that pretty prints a deps.edn dependency tree a la lein deps :tree? are people just using maven to accomplish that? #2019-06-07 16:06:31 zane You mean like clojure -Stree? #2019-06-07 16:06:56 mss exactly what I was looking for! thanks for the help #2019-06-07 16:07:21 zane You're welcome! #2019-06-07 17:34:42 Nolan I’ve got a java class on the classpath that looks like this:
public final class com.whatever.Main {
public static void main(java.lang.String[]);
}
Is there any way to use :main-opts to invoke com.whatever.Main.main()? I tried:
:aliases
{:some-alias
{:main-opts
["-m" "com.whatever.Main"]}}
but that looks for com/whatever/Main__init.class. I realize I’m probably stretching how aliases are meant to be used, but it would be mega convenient to be able to run this using the cli. #2019-06-07 17:48:59 zane You could create a Clojure namespace with a -main that just delegates to that Java method, maybe? 🤔 #2019-06-07 17:51:55 Nolan yeah, that’s probably the appropriate way to go about it. #2019-06-07 17:54:08 alexmiller clj is a Clojure launcher, not a general purpose Java launcher #2019-06-07 17:57:42 Nolan I subsequently realized how silly it was to be trying to use clj before there was a clojure component. Everything fell into place once I realized that the whole problem went away as soon as I started writing what I initially set out to write. #2019-06-08 22:47:56 kommen ran into a an unexpected situation #2019-06-08 22:48:05 kommen this is ok: #2019-06-08 22:48:18 kommen $ clj -Sdeps '{:deps {org.clojure/clojurescript {:git/url "" :tag "r1.10.520" :sha "0c353f1947089cb8b1f010b4294b94ac109d4ef6"} org.clojure/tools.reader {:mvn/version "1.2.2"}}}'
Checking out: at 0c353f1947089cb8b1f010b4294b94ac109d4ef6
Clojure 1.10.0
user=> (require 'clojure.tools.reader)
nil
user=> (doc clojure.tools.reader/read+string)
nil
user=> #2019-06-08 22:48:48 kommen but this should not work, as tools.reader 1.2.2 has no read+string: #2019-06-08 22:49:04 kommen $ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.520"} org.clojure/tools.reader {:mvn/version "1.2.2"}}}'
Clojure 1.10.0
user=> (require 'clojure.tools.reader)
nil
user=> (doc clojure.tools.reader/read+string)
-------------------------
clojure.tools.reader/read+string
([] [reader & args])
Like read, and taking the same args. reader must be a SourceLoggingPushbackReader.
Returns a vector containing the object read and the (whitespace-trimmed) string read.
nil
user=>
#2019-06-08 22:49:31 kommen turns out clojurescript is an uberjar which contains tools.reader 1.3.0, which takes precedence over the specified version in deps.edn #2019-06-08 22:50:06 kommen is that to be expected? #2019-06-08 23:12:24 alexmiller Yes, I think so #2019-06-08 23:13:02 alexmiller They vendor a couple things iirc, but asking in #cljs-dev would be better #2019-06-09 06:45:40 dominicm I think that clojurescript vendors an AOT version of t.r, which takes priority. #2019-06-10 09:07:14 martinklepsch I've found another potential bug/issue, let me know if it makes sense to file an issue.
With the code below we'll end up with two versions of Clojure on the classpath:
(tdeps/resolve-deps {:deps {'org.clojure/clojure {:mvn/version "1.9.0"}}
:mvn/repos default-repos}
{:extra-deps {"org.clojure/clojure" {:local/root "/Users/martinklepsch/.m2/repository/org/clojure/clojure/1.10.0-beta2/clojure-1.10.0-beta2.jar"}}})
It can be easily fixed by changing the "org.clojure/clojure" to be a symbol but I would expect tools deps to handle that(?) #2019-06-10 12:46:17 alexmiller you can file a ticket for that #2019-06-10 14:42:52 dominicm Feels like strings in that position should blow up #2019-06-10 14:44:02 dominicm However, strings would make a pretty neat reference to the canonical upstream for git repos, even when you're using a separate branch #2019-06-10 15:09:15 alexmiller it should be an error, libs are symbols #2019-06-12 14:12:54 unbalanced does anybody happen to know if there's a lein deps :tree equivalent for the command line tool? #2019-06-12 14:15:29 bronsa clj -Stree #2019-06-12 14:24:56 unbalanced you're awesome, thanks #2019-06-12 14:25:44 unbalanced does this also show conflicts? My datomic-pro seems to be cranky and I'm trying to track down why #2019-06-12 14:26:20 alexmiller it doesn't - you may want to post the message in #datomic #2019-06-12 14:26:35 alexmiller there are some common cases and someone might just know hte answer #2019-06-12 14:27:19 unbalanced aha. Thank ya sir #2019-06-12 15:56:27 unbalanced alright question... in deps.edn is there a way to specify username/password for servers? i.e., without doing this in ~/.m2/settings.xml:
<servers>
<server>
<id>my.datomic.com</id>
<username>username</username>
<password>password</password>
</server>
</servers>
#2019-06-12 16:00:12 unbalanced or perhaps another way to say this is, if I have :mvn/repos {"" {:url ""}} in my deps.edn and the ☝️ snippet in ~/.m2/settings.xml, and yet I'm still getting a 401 on my download for com.datomic/datomic-pro {:mvn/version "0.9.5786"} ... is there another way to accomplish the same thing or am I just doing it wrong? #2019-06-12 16:01:23 alexmiller that is the right way to go about it, so something must not be lined up correctly #2019-06-12 16:06:48 unbalanced okay, I'm stupid #2019-06-12 16:07:06 unbalanced I didn't realize in the examples that was just a snippet of settings.xml #2019-06-12 16:08:11 unbalanced need the full thing... working now #2019-06-12 16:10:23 alexmiller cool, it's a balance on how much to put without it being overwhelming in those docs #2019-06-12 16:21:41 dominicm I saw something really neat recently where you could expand an example into the fully working program, this would be a good use case for that #2019-06-12 17:26:51 sveri Yea, or a link to a full code example on github. #2019-06-12 16:14:18 unbalanced indeed #2019-06-13 19:53:19 jmv is it possible to set the location of downloaded artifacts? the CI we use treats each step in the build pipeline as a clean slate and so it re-downloads all of the dependencies for each step. #2019-06-14 02:31:27 alexmiller @jmv305 yes, set the :mvn/local-repo attribute at the root of deps.edn (or in -Sdeps on command line) - takes a string that is a path to the root #2019-06-14 14:51:52 jmv thank you alex, that worked! #2019-06-14 02:32:02 alexmiller clj -Sdeps '{:mvn/local-repo "/right/here"}' #2019-06-14 08:14:45 borkdude I’m getting an error when I put clojure 1.10.1 in my extra-deps. I wanted the dependency to be there in my maven repo during test time, since I’m analyzing the source code.
;; org.clojure/clojure {:mvn/version "1.10.1"}
#2019-06-14 08:15:07 borkdude The error being:
Running tests in #{"test"}
Syntax error (ClassNotFoundException) compiling at (clj_kondo/core_test.clj:1:1).
clojure.core$with_meta__5405
#2019-06-14 08:16:05 borkdude I didn’t want to declare that clojure version at my top level, since I don’t want to force that version on other people #2019-06-14 08:17:33 borkdude trying a repro #2019-06-14 08:22:40 borkdude I can force the download in another way, never mind #2019-06-14 09:51:42 delaguardo I'm using maven for that. clj -Spom and then mvn deploy. #2019-06-14 16:01:11 seancorfield Yup that's how I do it too. With depstar to create the JAR. #2019-06-14 16:03:26 seancorfield The pom generated by clj is minimal -- you'll need to add a bunch of stuff and edit things. See the pom.xml file in the depstar repo. Without all the extra stuff Clojars won't know how to refer back to your source repo and you won't be able to import your library on http://cljdoc.org either. #2019-06-14 16:03:59 seancorfield next.jdbc is also published this way. #2019-06-14 16:09:42 borkdude this is good to know. I currently still mirror my deps in a project.clj and then call lein deploy #2019-06-14 16:29:15 jmv do you update the version in the pom by hand? #2019-06-14 16:31:31 borkdude good question. I wonder where the version comes from since that’s not present in deps.edn? #2019-06-14 17:28:01 seancorfield Yes, I update the pom.xml <version> tag manually. But the the SCM <tag> is updated by my deploy shell script when I make a release. #2019-06-14 17:28:53 seancorfield So my process the first time is clj -Spom and add all the SCM stuff etc into the skeletal pom.xml (by copying from another project and editing all the URLs etc). #2019-06-14 17:31:22 seancorfield Subsequent times: do the edits to prepare for the release, push to GH, create a new release on GH with release notes, pull (to ensure I have the latest master locally), then clj -Spom && clj -A:jar project.jar && git status (to verify the pom.xml didn’t change again). Then I usually re-run the test suite at this point. Then deploy project, which updates the pom.xml from the HEAD SHA, commits & pushes it, then uses mvn deploy to push it up to http://clojars.org #2019-06-14 17:32:10 seancorfield This is how I update the pom.xml in my deploy script sha=`git rev-parse HEAD`
version=`fgrep '<version>' pom.xml | head -1 | sed 's;.*<version>\(.*\)</version>.*;\1;'`
echo "Setting tag to ${sha} for version ${version}..."
cp pom.xml /tmp/${1}.pom.xml
sed "s;<tag>[0-9a-f]*</tag>;<tag>${sha}</tag>;" < /tmp/${1}.pom.xml > pom.xml
#2019-06-14 17:32:31 seancorfield This is the deploy line: mvn deploy:deploy-file -Dfile=${1}.jar -DpomFile=pom.xml -DrepositoryId=clojars -Durl=
#2019-06-14 17:40:19 jmv thanks sean, this is helpful to see #2019-06-14 17:44:54 seancorfield However, it’s the <tag> in the pom.xml that drives http://cljdoc.org when it fetches the project from GH to analyze — and that means it gets the deployed version (which is one commit ahead of the release). Just something to bear in mind. #2019-06-14 17:45:56 jmv right that makes sense #2019-06-15 09:06:39 holyjak Thanks! #2019-06-15 11:20:02 sogaiu is anyone else using m4 or something similar to generate their deps.edn files? #2019-06-15 15:27:42 gerred 👋 if I'm working on a dependency from a local source, is there any way to re-load that dependency from the repl? #2019-06-15 18:02:25 orestis @gerred if you have a local source, can’t you just reevaluate the code in the REPL? #2019-06-17 16:28:14 gerred local source, but different repo. for example, working on an upstream lib but want to see the effects on whatever my downstream project is. 🙂 #2019-06-17 16:28:25 gerred otherwise, totally #2019-06-17 16:28:52 gerred (say, adding a feature to someone's upstream before I PR it) #2019-06-17 17:28:39 orestis Shouldn’t matter which repo the code is from, as long as your editor picks up the correct namespace (most do) #2019-06-16 10:45:30 borkdude @gerred yeah, you can do (require '[my.lib] :reload) #2019-06-16 11:11:23 minhnn how to auto reload new lib into repl when I add It to my deps.edn, It depend on IDE or I must do itt manualy with deps/resolve-deps and deps/make-classpath #2019-06-16 11:22:54 borkdude @minhnhat10bk there is add-lib in tools.deps, google for it #2019-06-16 11:43:07 minhnn I have check but It seem like no add-lib in clojure.tools.deps.alpha #2019-06-17 16:21:25 Travis Hey guys, I might be missing it in the docs but is there away with tools-deps to include java src in the project ? #2019-06-17 16:52:19 ghadi tools.deps doesn't compile Java @camechis #2019-06-17 16:54:00 Travis thanks, thats what I figured #2019-06-17 17:12:13 souenzzo clj idea is awesome. it should be available in every language. #2019-06-18 12:41:47 unbalanced Is there a way to download the dependencies without running the app? (use case, building Docker image) #2019-06-18 12:44:15 unbalanced clj -i? #2019-06-18 12:58:09 unbalanced ahh looks like clj -version will do the trick #2019-06-18 12:59:46 kirill.salykin clj -Stree? #2019-06-18 13:01:24 unbalanced yeah that's better ... clj -version isn't a thing #2019-06-18 13:01:39 unbalanced I was doing clj -e "1" but that felt stupid #2019-06-18 13:26:48 ghadi clojure -Spath > /dev/null #2019-06-19 12:12:52 vlaaad I heard some rumors Rich is thinking whether it should be in clojure core instead of tools-deps 😱 #2019-06-19 12:14:38 vlaaad I wish it is more official then a separate branch because it removes SO MUCH friction #2019-06-19 12:36:16 alexmiller We’ll get back to it at some point. Some part of it may migrate into core, hopefully that would be for 1.11 #2019-06-20 01:36:19 rgm Can I pull in new code from a :local/root dependency without restarting the repl? I’m trying to extract a lib and want to have my test runner on watch mode in the main project. #2019-06-20 01:37:46 rgm er, I guess now that I think about it that’s not quite the repl. I guess I mean is there something related to add-lib that I could run from the repl or a test namespace that can reload a known :local/root dep #2019-06-20 03:29:46 seancorfield @rgm Is that code you already have on the classpath, and it just got updated? Or is this a genuine addition of new code/dependencies? #2019-06-20 03:56:11 rgm It’s already on the class path. The plot twist is that there’s a fair bit of cljc and cljs. #2019-06-20 03:58:11 rgm Let’s imagine I have editor buffers open for both the main proj, and namespaces in the lib I’m extracting, nrepl’d or prepl’d in on the editor, and if I can avoid restarting the REPL jvm to get new lib code that’d be amazing and happy. #2019-06-20 04:00:26 seancorfield On the JVM, just eval code as you change it. No need for any reloading there. On the browser (cljs) side, that's what Figwheel/Shadow's hot reloading is all about. Or am I missing something @rgm? #2019-06-20 04:01:02 rgm Oh, you’re probably not. The idea that it might just work had genuinely not occurred to me. #2019-06-20 04:01:18 seancorfield (I keep a JVM REPL up and running for days, sometimes weeks, and never use a watcher or reloader process) #2019-06-20 04:01:43 rgm I didn’t imagine that figwheel would observe outside the project directory. #2019-06-20 04:03:34 seancorfield Hmm, if you're editing code outside the project as well, I don't know how Figwheel et al would cope. I was just reading about some editor/REPL setup tho' where you can have both JVM and browser REPLs running and when you eval clj it goes to the JVM, cljs goes to the browser, and cljc goes to both. #2019-06-20 04:04:13 seancorfield So with that workflow, you'd still follow a good REPL-Driven Development workflow of edit-eval, edit-eval, without even needing to save your files. #2019-06-20 04:04:31 seancorfield (I work for fairly long times across multiple files without saving sometimes) #2019-06-20 04:06:35 rgm Huh, yeah. Maybe if I evaluate the editor buffer via prepl that could work. #2019-06-20 06:48:40 Olical If you're still on Conjure that's <localleader>eb :) #2019-06-20 04:06:57 seancorfield I haven't touched cljs for over four years but I've made a public pledge to go back and spend this weekend getting familiar with modern tooling -- including Figwheel and Shadow -- although I don't know what editor I'll use for this learning project. I've moved away from nREPL (to "just" a Socket REPL or in-process REPL/REBL) but I think cljs will force me to use nREPL? #2019-06-20 04:48:19 rgm I’m having a pretty fun time with prepl and conjure (I’m a vimmer) … my experience in vim has been than nrepl + piggieback + fireplace.vim is still pretty rough. Colleagues using emacs/cider laugh at me and my struggles. #2019-06-20 05:15:06 seancorfield I've gone through a bunch of setups over the last nine years. I think I started with TextMate and a plain REPL... #2019-06-20 09:14:50 alexmiller Me too :) #2019-06-20 05:16:19 seancorfield ...tried a bunch of editor+repl combos, settled on Eclipse/CCW for a while, then moved (back) to Emacs -- I used it a lot 20-30 years ago, then switched to Atom/ProtoREPL and more recently Atom/Chlorine, which means I can get away from nREPL. #2019-06-20 05:17:00 seancorfield And, lately, I've been very enamored with REBL as part of my workflow so my normal work setup is clj/`deps.edn`, Atom/Chlorine, REBL with a Socket REPL. #2019-06-20 05:17:54 seancorfield But I've always had a soft spot for vi so I've been keeping an eye on Liquid, which is basically an in-process vi-like editing experience with built-in evaluation (and no REPL needed). #2019-06-20 05:19:04 seancorfield I have Liquid + REBL (started from clj/`deps.edn`) working now and will try to use that quite a bit going forward (but it does lack some parinfer/paredit/global-find'n'replace niceties). #2019-06-20 05:19:35 seancorfield I don't know how any of that works for cljs stuff tho'. Part of this weekend's exercise will be exploring tooling and figuring all that out. #2019-06-20 05:20:59 felipebarros You should totally be a streamer @seancorfield... I would love to watch you experimenting with that stuff and analyze your workflow. #2019-06-20 05:22:16 seancorfield @anantpaatra Yeah, I was very interesting in the #livestream channel when it started ages ago but the problem is I mostly work on a large (86K+ line) proprietary code base and couldn't stream that. #2019-06-20 05:22:41 seancorfield I am considering doing more screencasts (or even streaming) my OSS work tho'. #2019-06-20 05:23:14 seancorfield @anantpaatra did you see the 9 minute screencast I posted on YouTube of me working with Atom/Chlorine/REBL? #2019-06-20 05:39:34 felipebarros It was definitely interesting to watch that as it is a completely different workflow from everything I've seen so far. But the pace is like trying to appreciate a super-bike when it passes in front of you at 350km/h. You can hear it is loud, you can see it is expensive, it's probably beautiful and you quickly think "damn, I would love to ride that" but its gone before you even realize and you're left with your regular pedestrian day ahead of you. haha #2019-06-20 05:42:35 felipebarros I understand it was made for experienced developers though, but I would really like that clj/deps.edn details being slowly crunched. #2019-06-20 05:42:49 felipebarros And after I get that part, the REBL. #2019-06-20 05:50:37 seancorfield OK, good feedback. I didn't realize it came across like that. I thought it was slow and gentle. I'll try harder next time. #2019-06-20 05:23:57 felipebarros For example, watching someone going from zero to a complex project by using clj/`deps.edn` would be awesome. I truly need something like that. #2019-06-20 05:24:15 felipebarros I haven't! Going to check it now. #2019-06-20 05:24:15 didibus +1, not to mention you've got a great voice for it as well! #2019-06-20 05:24:34 seancorfield You'll be telling me next that I have an accent... 🙂 #2019-06-20 05:24:59 didibus 😛 #2019-06-20 05:26:24 seancorfield (I'm from the South of England. I don't have an accent. Everybody else does! 🙂 ) #2019-06-20 05:28:58 seancorfield BTW, I'm going to be on DEFN Sunday morning (Pacific) and I believe they're livestreaming that. But it's just chat, not coding. #2019-06-20 05:29:23 didibus Oh, cool. I'll be sure to tune in #2019-06-20 05:30:23 seancorfield When I fixed an issue on core.memoize last week, I realized that it would have been a great opportunity to stream or screencast. I used REBL and tap> specifically to hone in on the bug and fix it, all live-eval'ing in the editor. #2019-06-20 05:31:29 seancorfield (and we're a bit off-topic... -> #off-topic if we want to continue this!) #2019-06-20 07:44:11 sogaiu where are our black boxen when we need them? #2019-06-20 09:38:30 minikomi is there a better way? 🙂 #2019-06-20 09:41:50 alexmiller Not currently. There’s a ticket for it #2019-06-20 15:32:58 ghadi do you show the version of clj or the default version of clojure? or the requested version of clojure in the current context? #2019-06-20 19:33:29 orestis Everything? If it’s reasonably structured it can be greppable for tools to extract more. #2019-06-20 19:34:55 orestis I think Elixir’s mix and also npm (to which clj is more alike) show the version of the tool and the version of the language (and also the host runtime version). I’ll verify at a computer tomorrow. #2019-06-20 19:35:18 ghadi all i mean is the version of the language is context dependent #2019-06-20 19:35:38 ghadi (but the version of tool + jdk aren't) #2019-06-20 19:37:12 orestis Yeah for me the version of clj is more important - right now in my ansible scripts I’m looking at some some specific file to check a number. #2019-06-20 19:37:55 orestis Would prefer to invoke the tool and get an easily comparable version string without starting the JVM #2019-06-20 19:38:26 orestis (Not a big deal currently) #2019-06-20 21:02:05 alexmiller these questions are tbd, imo. I'm not sure that the subtle difference between the clojure version used when building classpaths vs the clojure version being used in your project is obvious to most people. I don't think it's even clear to people that these are independent things. #2019-06-20 21:18:18 vlaaad I just want to share how much I love tools-deps. It's so simple in what it does, you really know what's going on when you use it, and it's so easy to just specify git dep and immediately try something: no need to publish for library authors, no need to setup a project just to try stuff out for library users... I wish we had build tools that are that simple! #2019-06-20 21:18:51 alexmiller thanks! #2019-06-21 10:19:19 stathissideris does :jvm-opts work only within aliases or at the top-level too? #2019-06-21 10:20:34 kszabo only aliases, it’s a known issue #2019-06-21 10:20:47 stathissideris ok, thanks! (glad to hear it’s a bug and not a feature) #2019-06-21 17:45:46 misha 1) can I add a dependency to library so that it can be seen from library (e.g. when run as alias), but is invisible to client library? I assume deps in :aliases behave like that. and if so:
2) is there a way to depend on an alias of such a library?
Trying to have specs in my lib on one hand (for lib internal tests, and for lib's clients if they need them),
and as a lib client to be able to exclude them from dependency (in case I use another spec version, etc.) #2019-06-21 17:47:53 hiredman no #2019-06-21 17:48:34 misha no for both 1 and 2? #2019-06-21 17:48:58 hiredman yes #2019-06-21 17:49:09 misha what is the preferred way to go about it? separate repo with specs? #2019-06-21 17:49:57 misha do :aliases extra deps appear in client's class path? #2019-06-21 17:50:45 misha if no, that makes extra repo option not that bad #2019-06-21 17:50:53 hiredman I maybe have misunderstood 1 #2019-06-21 17:51:52 misha I want to have specs for my lib. And want you to be able to depend on my lib excluding those specs (and clojure.spec)
both at the same time #2019-06-21 17:51:54 seancorfield :aliases are not transitive across dependencies, if that's what you're asking @misha? #2019-06-21 17:52:40 hiredman so 1, the dep on clojure.spec is going to be there, clojure pulls it in #2019-06-21 17:53:23 misha so:
specs-repo +
lib-repo + specs-repo-as-alias-extra-dep for tests
is an ok solution #2019-06-21 17:53:51 hiredman or just put your specs in a namespace and don't load that namespace except from tests #2019-06-21 17:54:06 misha oh, right. then: can I not depend on clojure in my lib repo? ) #2019-06-21 17:54:08 seancorfield In separate repos, yes, I guess that could work -- but I don't know why anyone would bother? #2019-06-21 17:54:57 seancorfield If your specs are part of yours tests (only) just put them in the test folder that would be added by your :test alias. Done. #2019-06-21 17:55:08 misha trying to avoid any transitive deps you'll get by using my lib #2019-06-21 17:55:58 seancorfield What transitive deps would those be? Clients of your lib need whatever your src depends on. They don't need what your test code depends on. #2019-06-21 17:56:01 misha Sean, those specs might be useful for you as a client. But if you dont care, I dont want you to get clojure.spec through my lib #2019-06-21 17:56:07 hiredman spec comes with clojure these days, but is technically a separate artifact so you can use a newer version if you want #2019-06-21 17:56:12 seancorfield You can't avoid clojure.spec. #2019-06-21 17:56:46 seancorfield Clojure itself pulls in clojure.spec and core.specs. The client is always going to get those. #2019-06-21 17:57:10 misha I need specs for tests only, but I know those might be useful for clients as well. so core does not depend on those for functionality. #2019-06-21 17:57:13 seancorfield You can avoid test.check (by making it a :test-only :extra-deps). #2019-06-21 17:57:59 seancorfield @misha I think you're overthinking this. If you want clients to have access to your specs as an option, put them in src. If you don't want clients to have access to them, put them in test. #2019-06-21 17:58:13 misha ok. do I have to depend on clojure itself to distribute a lib? if yes, is it better to depend on as old version as possible? #2019-06-21 17:58:41 seancorfield The Clojure dependency will be overridden by the client. #2019-06-21 18:00:17 misha I am absolutely overthinking it, but on the other hand I had bad experience with different versions of direct and transitive deps before. trying to not mess things up for anyone. #2019-06-21 18:12:56 seancorfield Yeah, I hear that... If you have test-only dependencies, put them under an alias as :extra-deps. If you have source dependencies, those are going to end up as transitive dependency for your clients. #2019-06-21 18:29:28 misha You are right, clojure includes spec, I have to include clojure, so it boils down only to "make specs public or not". thanks, guys. #2019-06-21 18:30:59 seancorfield As an example, both clojure.java.jdbc and next.jdbc have all their specs in a dedicated namespace that clients can require if they wish but otherwise they are not activated. #2019-06-21 18:32:54 seancorfield For clojure.java.jdbc, I primarily made them optional like that so it could continue to run on older Clojure versions (that didn't support Spec). For next.jdbc, I added them after the fact as an option for folks who want better error messages (since next.jdbc sort of follows Clojure core's principle of "do the right thing as fast as possible; potentially explode in arbitrary ways if asked to do the wrong thing" (i.e., NPE or ClassCast or whatever). #2019-06-21 18:48:24 misha this is exactly how I am approaching it now. Docstrings – contract, validation – optional, in a separate ns #2019-06-21 19:45:55 andy.fingerhut Fun fact. This works: clj -Sdeps "{:deps {org.clojure/clojure {:mvn/version \"1.0.0\"}}}" #2019-06-21 19:47:13 andy.fingerhut And if you are wondering, I tried it because it is helpful in tracking down when some functions were added to Clojure that do not (yet) have :added metadata, but should soon. #2019-06-21 19:53:53 seancorfield Interesting little glitch in 1.0.0: (! 563)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.0.0"}}}'
Clojure 1.0.0-
user=> (clojure-version)
"1.0.0-"
user=> *clojure-version*
{:major 1, :minor 0, :incremental 0, :qualifier ""}
user=> ^D
Fri Jun 21 12:53:28
(sean)-(jobs:0)-(~/clojure)
(! 564)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.1.0"}}}'
Clojure 1.1.0
user=> (clojure-version)
"1.1.0"
user=> *clojure-version*
{:major 1, :minor 1, :incremental 0, :qualifier ""}
user=>
#2019-06-21 19:55:49 andy.fingerhut I'll file a ticket for a 1.0.1 release 🙂 #2019-06-21 20:04:20 alexmiller Right :) #2019-06-22 22:21:44 seancorfield A screencast about the CLI, deps.edn, and aliases https://youtu.be/CWjUccpFvrg (this time in 1080p instead of 360p!). #2019-06-23 06:53:53 felipebarros Thank you so much for that! The pace is elegant, the scope is healthy and the perspective is intimate and clarifying. Excellent! #2019-06-23 05:29:22 steveb8n Q: I can’t get :override-deps to work with :local/root deps. e.g. I have a default :git/sha dep and then for local dev I want to override that with a :local/root dep. Has anyone else made this work? #2019-06-23 05:34:23 steveb8n starting a separate thread to stop annoying others #2019-06-23 05:35:04 seancorfield This sounds like it should work, but I’d have to see the relevant parts of the deps.edn file I think… #2019-06-23 05:35:39 steveb8n my specific problem is that the the :git/sha dep is not reachable in local dev so I’m trying to override it. however, with the override, it still tries to pull the :git/sha dep and that breaks #2019-06-23 05:36:35 steveb8n I agree. it should work. however it appears that with an :override-deps, the overridden dep must still be accessible #2019-06-23 05:37:21 steveb8n let me see if I can make a simple test case deps.edn to demo it #2019-06-23 05:39:36 seancorfield I tried from the command line with an alias to provide override-deps and it seems to work… #2019-06-23 05:41:05 seancorfield Yup, deleted my ~/.gitlibs folder and the :dev alias that overrides the dep with :local/root works fine. No attempt is made to clone the git repo. #2019-06-23 05:44:24 steveb8n ok, here’s a repro (I think) #2019-06-23 05:44:35 steveb8n {:deps {reifyhealth/specmonstah {:mvn/version "2.0.0-alpha-1"}}
:aliases {:local {:override-deps {reifyhealth/specmonstah {:git/url ""
:sha "8e47ffc60d3177cea90f2f657e73c48971d8a1e4"}}}}}
#2019-06-23 05:45:03 steveb8n yuck formatting #2019-06-23 05:45:54 steveb8n then clojure -R:local -Spath | tr ‘:’ ‘\n’ | sort #2019-06-23 05:46:28 steveb8n shows the alpha version, not my fork #2019-06-23 05:46:57 steveb8n not quite the same as what I described but I think it might show how I am using the CLI opt wrong #2019-06-23 05:50:56 steveb8n hmm, I wonder if my version of tools.deps is too old and this is a fixed bug. I’ll check that now #2019-06-23 05:51:20 seancorfield (! 640)-> clj -Spath | tr ':' '\n' | sort| fgrep specmon
/Users/sean/.m2/repository/reifyhealth/specmonstah/2.0.0-alpha-1/specmonstah-2.0.0-alpha-1.jar
Sat Jun 22 22:50:18
(sean)-(jobs:0)-(~/clojure/steve)
(! 641)-> clj -A:local -Spath | tr ':' '\n' | sort | fgrep specmon
/Users/sean/.gitlibs/libs/reifyhealth/specmonstah/8e47ffc60d3177cea90f2f657e73c48971d8a1e4/src
Sat Jun 22 22:50:24
(sean)-(jobs:0)-(~/clojure/steve)
(! 642)-> cat deps.edn
{:deps {reifyhealth/specmonstah {:mvn/version "2.0.0-alpha-1"}}
:aliases {:local {:override-deps {reifyhealth/specmonstah {:git/url ""
:sha "8e47ffc60d3177cea90f2f657e73c48971d8a1e4"}}}}}
works for me #2019-06-23 05:51:39 seancorfield -R:local behaves the same, I checked. #2019-06-23 05:51:53 seancorfield (! 644)-> clj -Sdescribe
{:version "1.10.1.447"
#2019-06-23 05:53:01 steveb8n I’m 1.9.0.391 #2019-06-23 05:53:22 steveb8n doesn’t seem likely it’s a bug but I guess I should rule that out #2019-06-23 05:54:01 seancorfield Yeah, I think this is a bug that was fixed (ages ago). #2019-06-23 05:54:58 steveb8n yeah. I just ran the same commands you pasted in and local does not change the result #2019-06-23 05:55:05 steveb8n I’ll trying upgrading #2019-06-23 05:55:29 steveb8n not sure how 😉 #2019-06-23 05:55:39 steveb8n I’ll go find the docs for upgrading #2019-06-23 05:55:39 seancorfield Mac or Linux? #2019-06-23 05:56:43 steveb8n mac. I think it’s “brew upgrade clojure” #2019-06-23 05:56:45 steveb8n trying that now #2019-06-23 05:57:02 seancorfield Yup, that’s the right command. #2019-06-23 05:57:10 steveb8n waiting on Aussie broadband 🙂 #2019-06-23 05:57:56 seancorfield Sorry. Mine’s about 150M down and 7.5M up. #2019-06-23 05:58:58 steveb8n mines 94m down which is really good for Aussie. I can’t complain. I suspect the latency hurts despite the good bandwidth #2019-06-23 05:59:30 seancorfield Yah, we have global monitors for our work stuff and the latency from Asia/Australia is pretty bad. #2019-06-23 05:59:55 seancorfield (our main data center is US East Coast and our primary AWS region is also US East Coast) #2019-06-23 06:02:53 steveb8n ok. that fixed it in the repro case! brilliant #2019-06-23 06:03:00 steveb8n let me check my original #2019-06-23 06:03:25 steveb8n yep, fixed there too #2019-06-23 06:03:47 steveb8n thanks for being a sounding board/tester 🙂 #2019-06-23 06:03:53 steveb8n I’ll try to pay it forward #2019-06-23 06:03:59 seancorfield 1.9 is super old. So many changes, fixes, enhancements since then! #2019-06-23 06:04:24 seancorfield (but then I’m the guy who always runs Clojure alpha/beta builds in production! LOL!) #2019-06-23 06:04:29 steveb8n yeah. I’m using 1.10 for my project, just not for the CLI that kicks it all off #2019-06-23 06:04:38 steveb8n now I am 🙂 #2019-06-23 06:05:36 steveb8n have a good day/night. and thanks again for the videos, I’m gonna go watch the rest of them now #2019-06-23 06:14:16 seancorfield G’nite! It’s 11:15pm here and definitely bedtime. #2019-06-23 05:30:06 steveb8n btw: @seancorfield I just watched your video on deps.edn. really useful. thanks. #2019-06-23 05:33:54 seancorfield Glad it was helpful! #2019-06-23 05:34:12 seancorfield Can you share your deps for the git/url and local/root deps in the alias? #2019-06-23 19:34:52 didibus Is there a way to specify jvm options but not for an alias? #2019-06-23 19:35:15 didibus And not at the command line #2019-06-23 19:35:31 didibus So have them specified in deps.edn for when I run main #2019-06-23 20:01:47 seancorfield Right now they only work inside an alias @didibus — it’s a known issue/limitation. Sounds like the team may change that at some point… #2019-06-23 20:02:07 didibus I see, thx ! #2019-06-23 21:41:13 andy.fingerhut Does anyone happen to know when you make a text file beginning with #! /usr/bin/env clojure on a Unix/Linux system, once the JVM begins and the Clojure reader starts reading the file, is the Clojure reader also given that first line to read as well? #2019-06-23 21:41:48 andy.fingerhut Or is there some code somewhere that somehow skips over it, and thus the Clojure reader never sees it? #2019-06-23 21:42:22 andy.fingerhut And if so, do you know where that "skipping" code is? #2019-06-23 21:49:39 alexmiller In LispReader #2019-06-23 21:50:07 alexmiller It skips lines that start with #! #2019-06-23 21:50:31 alexmiller Or rather, those act as single line comments #2019-06-23 21:54:07 alexmiller It was certainly a future looking escape in the reader :) #2019-06-23 21:57:06 andy.fingerhut And I have a vague feeling that I have asked this and/or discovered it before, and was also then surprised that it has been around since 2009 #2019-06-23 21:59:22 andy.fingerhut Do you think it belongs in the Reading Clojure Characters guide, or better to keep it obscure? 🙂 #2019-06-23 22:09:27 alexmiller Well I don’t think it belongs there as it’s not really part of the Clojure language #2019-06-23 22:10:03 alexmiller Really more syntax intentionally ignored by Clojure #2019-06-23 22:10:23 alexmiller Could be on the reader page #2019-06-23 23:30:24 andy.fingerhut I haven't followed the myriad ways people have used deps and clj/clojure, but does the approach on that page: https://gist.github.com/ericnormand/6bb4562c4bc578ef223182e3bb1e72c5 pretty much a good way to go, if you want to embed the JVM and/or clj/clojure options within the same file as the Clojure code? As in, have people found other ways to combine those options together into the same file as the Clojure code, so that those options do not end up in a separate file/script somewhere? #2019-06-24 12:08:30 futuro I love the use of #_ to hide the shell opts from the Clojure reader. #2019-06-24 16:18:41 andy.fingerhut It does restrict you a little bit, in that those first 16 lines must be valid shell syntax, as well as legal Clojure syntax, but yeah, I am pretty sure it is not the first language where someone has found a handful of dual-language lines of code. #2019-06-24 17:33:26 futuro Hmm, that’s a good point. #2019-06-24 23:23:15 rgm is the new jira taking new commenters yet? Wanted to update TDEPS-91 #2019-06-24 23:33:03 andy.fingerhut The new JIRA does allow adding comments, if you have a free account created, and if that account's permissions have been bumped up by a Clojure JIRA admin. #2019-06-24 23:34:15 rgm oh, cool… is that someone less busy than Alex? #2019-06-24 23:34:24 andy.fingerhut At least, the older Clojure JIRA instance required someone to bump up your permissions. My permissions allow me to do so with the new JIRA, but I believe Alex bumped up most/all people who have created or commented on issues in the old JIRA, in the new one. #2019-06-24 23:35:06 andy.fingerhut I had permission to do that on the old JIRA system, but not sure whether I do on the new one or not. Let me take a look. Otherwise, I suspect it would not be someone less busy than Alex. #2019-06-24 23:36:11 rgm no rush; I have enough of a handle on what’s going on in ~/.gitlibs that I’ve just manually set it up so clj et al don’t complain any more, and I can do manual git pulls for a while. #2019-06-24 23:38:20 rgm just wanted to bump this since it’s gonna bite a lot of people come 10.15 release in September. #2019-06-24 23:39:01 andy.fingerhut To be clearer, I mean that my account on the new Clojure JIRA gives me permission to add comments on issues. I do not know how on the new JIRA to give permission to others to do so, and/or my account on the new JIRA doesn't have authorization to do so. #2019-06-24 23:39:15 rgm ah, gotcha. I’ll ask Alex. #2019-06-25 01:00:39 ghadi @rgm an id_rsa key should work just fine. Are you running under an ssh agent? #2019-06-25 01:01:02 rgm I am. #2019-06-25 01:01:42 ghadi is the key inside your agent? #2019-06-25 01:01:54 ghadi ssh-add -l #2019-06-25 01:02:04 ghadi should verify ^ #2019-06-25 01:05:40 ghadi I've edited TDEPS-91 to be problem-oriented rather than leading with a solution #2019-06-25 01:14:35 rgm yep, it is. #2019-06-25 01:14:57 rgm the same key file works under 10.14 and fails under 10.15, same clj version #2019-06-25 01:15:08 rgm under an agent both places. #2019-06-25 01:21:26 ghadi if you can find out what the difference is, I'd be much obliged #2019-06-25 01:21:33 ghadi is SSH_AUTH_SOCK set? #2019-06-25 01:50:55 rgm hm, good question. It appears yes. #2019-06-26 15:07:16 credulous Hi! I’m trying to convert my workflow to clj/deps.edn, and I’m having a bit of trouble with depstar, which I am trying to use to package my app. #2019-06-26 15:07:35 ghadi seancorfield/depstar @credulous? #2019-06-26 15:07:56 ghadi can you describe what you expected and what you are experiencing? #2019-06-26 15:08:14 credulous healthfinch/depstar I think? #2019-06-26 15:08:29 credulous Yes I can… I added this to my ~/.clojure/deps.edn: #2019-06-26 15:08:33 credulous :depstar {:extra-deps {com.healthfinch/depstar {:git/url ""
:sha "4aa7b35189693feebc7d7e4a180b8af0326c9164"} }
:main-opts ["-m" "hf.depstar.uberjar"]
} #2019-06-26 15:08:51 ghadi switch to the fork (I originally wrote healthfinch/depstar but now sean has the canonical repo) #2019-06-26 15:08:59 credulous OK! #2019-06-26 15:09:06 credulous I’ll try that before I describe anything more. #2019-06-26 15:09:07 credulous Thanks #2019-06-26 15:09:17 ghadi his README is good, too #2019-06-26 15:09:21 ghadi better #2019-06-26 15:09:27 credulous In his blog post, Sean is still linking to yours #2019-06-26 16:00:25 seancorfield I'll update that blog post when I get to my desk -- thanks for catching that! #2019-06-26 17:01:57 seancorfield Updated. Make take a few minutes to clear the cache! #2019-06-26 15:09:42 ghadi which blog post? #2019-06-26 15:13:09 credulous Yup, the fork works. Thanks @ghadi! #2019-06-26 15:15:48 credulous And may I also say: @seancorfield is everywhere! What a contributor. #2019-06-26 15:18:38 ghadi he is a treasure #2019-06-26 15:20:53 alexmiller MVP! #2019-06-26 17:01:51 credulous Thanks! Also, I’m floundering a bit trying to include my public folder in the uberjar, using depstar. I created an alias with {:extra-paths ["public"]}, but that includes the contents of the public folder into the root of the uberjar. Any advice? #2019-06-26 17:02:47 seancorfield @credulous So you have resources/public -- would {:extra-paths ["resources"]} do what you need? #2019-06-26 17:03:19 credulous No, in my project folder I have /resources and /public #2019-06-26 17:03:27 seancorfield Don't do that 🙂 #2019-06-26 17:03:30 credulous OK! #2019-06-26 17:03:50 seancorfield Put public inside resources -- then your "source" paths are going to be "src" "resources" #2019-06-26 17:04:18 seancorfield And in your code, specify that pages etc come from "public" rather than the root of your classpath. #2019-06-26 17:05:31 credulous Sounds good. I was cribbing from a complete project on github… boodle, I think. #2019-06-26 17:05:40 seancorfield {:root "public"} in Compojure's resources call (assuming you're using that?) #2019-06-26 17:07:07 credulous I’m probably using compojure wrong too… I’m not using :root anywhere #2019-06-26 17:07:13 credulous (compojure/defroutes app
(-> (compojure/routes
(route/resources "/")
api/routes )
reload/wrap-reload))
#2019-06-26 17:07:42 credulous where route/resources defaults to “public” as the source directory, I think #2019-06-28 10:36:20 vlaaad Can someone explain how do I build a classpath properly?
(deps/make-classpath
(deps/resolve-deps {:deps {'re-find {:git/url ""
:sha "c27af619a9d9e12832b7446e127d68ab916ce78c"}}}
nil)
nil
nil)
produces this classpath:
- .../tools.cli-0.4.1.jar
- .../math.combinatorics-0.1.4.jar
- .../clojure-1.8.0.jar"
but there should also be .gitlibs/libs/refind/...sha/src, no? #2019-06-28 11:54:02 vlaaad Apparently I had to add :paths ["src"] to map, which works, but feels a bit wrong... shouldn't it be the default? if I'll set :paths to ["src/clj"] in my ~/.clojure/deps.edn, will it break dependencies because I overridden their source path? #2019-06-28 11:54:11 vlaaad Apparently I had to add :paths ["src"] to map, which works, but feels a bit wrong... shouldn't it be the default? if I'll set :paths to ["src/clj"] in my ~/.clojure/deps.edn, will it break dependencies because I overridden their source path? #2019-06-28 12:54:49 alexmiller :paths ["src"] is in the install-level deps.edn, but when you're using the library directly, you're in full control of what you include #2019-06-28 12:55:47 alexmiller setting to "src/clj" only affects your lib, not transitive deps - they set their own paths in their own deps.edn #2019-06-28 12:59:47 vlaaad that's the thing, re-find does not set :paths explicitly, that's why I need to specify ["src"] #2019-06-28 13:00:14 vlaaad it all works until everyone's system :paths property is the same #2019-06-28 13:00:54 vlaaad setting "src/clj" affects dependent libs too #2019-06-28 13:03:05 vlaaad (run! println
(str/split
(deps/make-classpath
(deps/resolve-deps
{:deps {'re-find {:git/url ""
:sha "c27af619a9d9e12832b7446e127d68ab916ce78c"}}
:paths ["src/clj"]}
nil)
nil
nil)
#":"))
=>
/home/vlaaad/.gitlibs/libs/re-find/re-find/c27af619a9d9e12832b7446e127d68ab916ce78c/src/clj
/home/vlaaad/.m2/repository/org/clojure/tools.cli/0.4.1/tools.cli-0.4.1.jar
/home/vlaaad/.m2/repository/org/clojure/math.combinatorics/0.1.4/math.combinatorics-0.1.4.jar
/home/vlaaad/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar
#2019-06-28 13:05:35 vlaaad re-find sources are implicitly on src, not src/clj, but it gets overridden #2019-06-28 13:15:17 alexmiller sorry, I don't understand what you're saying #2019-06-28 13:16:18 alexmiller it looks like answer above is what I would expect to see #2019-06-28 13:16:24 alexmiller it looks like answer above is what I would expect to see #2019-06-28 13:18:44 vlaaad I am surprised that I have to specify :paths to get code from git dependency #2019-06-28 13:19:51 vlaaad I expect gitlibs to have default :paths ["src"] #2019-06-28 13:22:23 alexmiller oh, I see what you're saying #2019-06-28 13:23:25 alexmiller it's up to libs to specify their own paths and re-find could (probably should) set :paths ["src"], and if it did, that would be used #2019-06-28 13:23:57 alexmiller you're seeing src/clj there due to a bug, where if no paths are used, the paths in the root deps.edn are used #2019-06-28 13:36:09 vlaaad ah, it's already reported, nice #2019-06-28 13:39:51 alexmiller yeah, the cause is bound up in some other issues so I've had a long mull about how to fix, but ready to move forward on it when I have some time #2019-06-28 14:22:17 vlaaad clojure.tools.deps.alpha.specs seem wrong #2019-06-28 14:22:25 vlaaad (s/def :local/root string?)
(s/def :local/coord (s/keys :req-un [:local/root] :opt-un [::path]))
#2019-06-28 14:22:39 vlaaad shouldn't it be :req instead of :req-un? #2019-06-28 14:35:03 alexmiller yes, I think you're right #2019-06-28 14:35:56 alexmiller fixed #2019-06-28 21:33:08 drewverlee I'm attempting to build an uberjar using deps + uberdeps Everything works as expected until i try to run the uberjar and it tells me it cant find clojure.main.
➜ cat deps.edn
{:aliases {
:uberjar {:extra-deps {uberdeps {:mvn/version "0.1.4"}}
:main-opts ["-m" "uberdeps.uberjar"]}}}
➜ clj -A:uberjar
[uberdeps] Packaging target/frog.jar...
[uberdeps] Packaged target/frog.jar in 2 ms
➜ cat src/core.clj
(ns frog.core
(:gen-class))
(defn -main [& args]
(println "HIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII"))
➜ java -cp target/frog.jar clojure.main -m frog.core
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
As a starting point, im not sure why the instructions from uberdeps have me look for clojure.main, if its an uberjar isn't it included? #2019-06-28 21:35:25 hiredman if you don't have clojure in your deps #2019-06-28 21:35:50 hiredman some of it depends on how smart the uberdeps library is #2019-06-28 21:36:30 hiredman if you don't include a dep on clojure explicitly, you will fallback to whatever is in the system (or whatever it is called) deps.edn file that was installed when you installed clj #2019-06-28 21:37:03 hiredman but uberdeps might not be processing your deps.edn file using the tools.deps code, so it might not know to do that fallback #2019-06-28 21:37:29 hiredman so if you don't get include a clojure version in your deps.edn, there wouldn't be clojure in your jar #2019-06-28 21:37:55 hiredman yep #2019-06-28 21:38:02 hiredman it just slurps in the edn and reads it #2019-06-28 21:38:54 hiredman a lot of tools built on top of deps.edn are super immature and very broken, and they largely duplicate each other #2019-06-28 21:39:39 hiredman this is the second broken uberjar building library I have diagnosed today (the other tool seemed to end up aot compiling things twice) #2019-06-28 21:41:54 hiredman also, if you are not aot compiling you don't need the :gen-class #2019-06-28 22:08:24 drewverlee I thought it was necessary to for the rest of the tooling to convert the clojure code to java. #2019-06-28 22:16:30 drewverlee Oh, your saying that if I don't specify that, the compiling still happens, just not ahead of time? #2019-06-28 22:19:13 hiredman clojure code is not converted to java, it is compiled to jvm bytecode #2019-06-28 22:20:04 hiredman it is always compiled to jvm bytecode, either when the code is loaded and right before it is run, or you load the code ahead of time and save the compiled bytecode and run the compiled bytecode (aot compilation) #2019-06-28 22:20:20 hiredman :gen-class only does anything when you are aot compiling #2019-06-28 22:20:45 hiredman and it is only needed in specific circumstances #2019-06-28 22:20:57 hiredman you are not aot compiling so you do not need it #2019-06-28 22:31:06 drewverlee Almost understand what your saying, I'll need to read a bit. Thanks! #2019-06-28 21:44:50 alexmiller a new version of tools.deps is out now that probably makes it much harder to do the wrong thing with the install deps.edn #2019-06-28 21:45:15 hiredman I dunno, this code literally just slurps the deps.edn and calls read-string on it #2019-06-28 21:45:23 seancorfield @drewverlee Try using depstar instead of uberdeps -- unless there's some specific feature you need from the latter? #2019-06-28 21:49:45 drewverlee There isn't! I did try both, I think I got a similar error. For one, It's possible I have an old version of deps. I'll have to check when I'm at my computer again. I'm always blown away by how fast everyone responds. #2019-06-28 21:50:57 seancorfield Happy to help with any issues you have with depstar since I maintain it. #2019-06-28 23:46:03 drewverlee So it resulted in a FileNotFoundException:
➜ cat deps.edn
{:deps {org.clojure/clojure {:mvn/version "1.10.0"}}
:aliases {:depstar
{:extra-deps
{seancorfield/depstar {:mvn/version "0.2.1"}}}}}
➜ clojure -A:depstar -m hf.depstar.uberjar MyProject.jar
Building uber jar: MyProject.jar
➜ frog java -cp MyProject.jar clojure.main -m frog.core
Exception in thread "main" java.io.FileNotFoundException: Could not locate frog/core__init.class, frog/core.clj or frog/core.cljc on classpath.
at clojure.lang.RT.load(RT.java:466)
at clojure.lang.RT.load(RT.java:428)
at clojure.core$load$fn__6824.invoke(core.clj:6126)
at clojure.core$load.invokeStatic(core.clj:6125)
at clojure.core$load.doInvoke(core.clj:6109)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5908)
at clojure.core$load_one.invoke(core.clj:5903)
at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
at clojure.core$load_lib.invokeStatic(core.clj:5947)
at clojure.core$load_lib.doInvoke(core.clj:5928)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$load_libs.invokeStatic(core.clj:5985)
at clojure.core$load_libs.doInvoke(core.clj:5969)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$require.invokeStatic(core.clj:6007)
at clojure.main$main_opt.invokeStatic(main.clj:491)
at clojure.main$main_opt.invoke(main.clj:487)
at clojure.main$main.invokeStatic(main.clj:598)
at clojure.main$main.doInvoke(main.clj:561)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:37)
➜ clojure -A:depstar -m hf.depstar.uberjar MyProject.jar
Building uber jar: MyProject.jar
➜ java -cp MyProject.jar clojure.main -m frog.core
Exception in thread "main" java.io.FileNotFoundException: Could not locate frog/core__init.class, frog/core.clj or frog/core.cljc on classpath.
at clojure.lang.RT.load(RT.java:466)
at clojure.lang.RT.load(RT.java:428)
at clojure.core$load$fn__6824.invoke(core.clj:6126)
at clojure.core$load.invokeStatic(core.clj:6125)
at clojure.core$load.doInvoke(core.clj:6109)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5908)
at clojure.core$load_one.invoke(core.clj:5903)
at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
at clojure.core$load_lib.invokeStatic(core.clj:5947)
at clojure.core$load_lib.doInvoke(core.clj:5928)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$load_libs.invokeStatic(core.clj:5985)
at clojure.core$load_libs.doInvoke(core.clj:5969)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:667)
at clojure.core$require.invokeStatic(core.clj:6007)
at clojure.main$main_opt.invokeStatic(main.clj:491)
at clojure.main$main_opt.invoke(main.clj:487)
at clojure.main$main.invokeStatic(main.clj:598)
at clojure.main$main.doInvoke(main.clj:561)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:37)
➜ cat src/core.clj
(ns frog.core)
(defn -main [& args]
(println "Hello World"))
➜ clj -Spath
src:/home/drew/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0.jar:/home/drew/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/drew/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
I assume because as src/ is on my class path (which is default right?). Then it would pick up the core.clj file with the frog.core ns. But either it doesn't pick it up, or i'm not specifying it correctly.
It seems to be in the jar file:
#2019-06-28 23:46:03 drewverlee ➜ cat core.clj
(ns frog.core)
(defn -main [& args]
(println "Hello World"))
#2019-06-29 00:02:37 seancorfield Is that really src/core.clj or src/frog/core.clj? #2019-06-29 00:03:08 seancorfield (it should be the latter: the file path must match the namespace segments!) #2019-06-29 00:03:50 seancorfield A good sanity check is to see whether clj -m frog.core works before you try to build the JAR. #2019-06-29 00:07:11 drewverlee ah ok. i guess i didn't realize the relationship between the files and the ns was more then convention. #2019-06-29 00:22:06 seancorfield When you ask Clojure to run frog.core, how is it going to find it? The classpath is all file paths, so the ns has to be mapped to a file path to perform that search if it can't already find the compiled class in memory... so that's frog/core.clj (or frog/core.cljc) relative to each folder on the classpath, e.g., src. #2019-06-28 21:45:30 alexmiller well, if you're not using tools.deps at all, then yes, that's prob broken #2019-06-28 21:45:57 seancorfield Some of these packaging libs try to create a classpath which is not what you would have when running your code 😞 #2019-06-28 21:46:06 alexmiller but I would encourage whomever to file tickets on these libs so they can be better, rather than raging about brokenness #2019-06-28 21:46:37 seancorfield (I hadn't even heard of uberdeps before just now) #2019-06-28 21:47:19 hiredman I could be wrong too, I am just guessing at the problem #2019-06-28 21:51:26 hiredman I am just suggesting, like, be aware of that state of maturity of a lot of these libraries(tools.deps is still new, so tooling built on it is even newer), and maybe vet them a little more before using them. #2019-06-28 21:52:30 andy.fingerhut Well, vetting includes using, at least a little bit 🙂 #2019-06-29 08:31:30 dominicm I'm fairly confident that pack does the right thing™. We had some early issues due to the system deps.edn (although it sounds like I'm able to remove that now! 🎉). I think pack is the oldest library in this space, and I've been incredibly careful about things like classpath collisions, which other tools have been more unconcerned about. #2019-06-29 11:37:53 jose I just installed clojure 1.10.1.455, using https://download.clojure.org/install/linux-install-1.10.1.455.sh. Running clj command starts a REPL with clojure 1.9, while if I install the previous version, 1.10.447, clj command starts a REPL with clojure 1.10.1.
I think the reason is that in the latest released jar, at clojure/tools/deps/deps.edn, clojure version is set to 1.9 and that I don't require a clojure version on my ~/.clojure/deps.edn
Is this the intended behavior? #2019-06-29 12:20:04 alexmiller Oh, that is a bug #2019-06-29 12:20:26 alexmiller I know exactly why that’s happening #2019-06-29 12:20:30 alexmiller Thx #2019-06-29 12:38:28 alexmiller @jlle 1.10.1.458 is now available, should fix that #2019-06-30 08:38:26 jose Tested it, now clj start a REPL with 1.10.1. Thanks for fixing it! #2019-06-29 14:47:45 alexmiller For the toolsmiths out there, I'm going to deprecate the reader/clojure-env call in tools.deps.alpha (and eventually remove it). Afaik, this was being used exclusively to obtain the list of config files used by clj. As of the latest tools.deps.alpha, 0.7.516, there is a new call reader/default-deps that will return the list of deps.edn config files. The install-level deps.edn is now embedded and this function replicates the logic from clj to find the user-level deps.edn and project-level deps.edn without shelling out (so also OS-safe). I sent a few PRs to places where I could find people doing this. @dominicm @olical @seancorfield @taylor #2019-06-29 15:08:30 Olical Wonderful, thank you very much! #2019-06-29 15:05:36 dominicm I think that I embedded my own copy of the system deps.edn a while ago. I'll make an issue to remove it and switch to whatever I'm supposed to be doing now. #2019-06-29 15:08:27 alexmiller lmk if you have any questions #2019-06-29 15:11:58 dominicm looks like the read-deps call in pack can now be a direct call to t.d.a, I was doing something very similar with a slurp to the deps.edn file. #2019-06-29 15:12:57 dominicm quite ot, but why did you ^:const the path to the install deps.edn? As in, what made you think to do it? j/w, I've never really seen code use ^:const. Oh, is it because of optimizing performance in t.d.a as much as possible in order to minimize startup time? #2019-06-29 15:20:40 alexmiller it is literally a string constant, so it can be inlined #2019-06-29 15:21:13 alexmiller I think there is no chance of any perf difference you'd notice :) #2019-06-29 18:46:08 alexmiller @vlaaad this fixes the thing you pinged about yesterday #2019-06-29 18:48:16 vlaaad Nice, thanks! #2019-06-30 01:53:49 seancorfield Nice round of bug fixing on t.d.a.! Inspired me to bash a bunch of bugs in clj-new and to cut new releases of depstar and clj-new 🙂 #2019-06-30 01:54:53 alexmiller Yep, felt good to do finally #2019-06-30 01:55:21 alexmiller I’m trying to decide if there’s anything left that’s critical before pulling out of alpha #2019-06-30 01:56:36 seancorfield TDEPS-91 ? #2019-06-30 01:58:16 seancorfield I'm a bit surprised that TDEPS-132 isn't fixed by some of the changes in 0.7.516? Or is that just waiting for @dominicm to see what state that ticket should be in with the latest version? #2019-06-30 01:59:51 alexmiller The git stuff is all independent of api. Plenty of things to fix/add still, just don’t think they will affect the api #2019-06-30 02:01:06 alexmiller I haven’t looked at 132, not sure what it needs still #2019-06-30 02:02:20 alexmiller But don’t think it affects api #2019-06-30 02:04:28 seancorfield Ah, OK. Yeah, I guess TDEPS-91 is pure implementation. #2019-06-30 02:04:58 seancorfield What about add-lib at this point? #2019-06-30 02:07:26 seancorfield Will you be updating the add-lib branch from master @alexmiller? #2019-06-30 02:08:52 seancorfield (I'm toying with the idea of doing a screencast of building a small web app from scratch but was thinking of using add-lib to show how to add dependencies while you're working live in an app... thoughts?) #2019-06-30 02:09:58 alexmiller add-lib is additive so, wouldn’t affect alpha. Not sure yet where that will go #2019-06-30 02:10:28 alexmiller I can update add-lib if you need it, although it’s not far behind master #2019-06-30 02:11:32 seancorfield It would be nice to keep it current with each release "just because" 🙂 #2019-06-30 04:12:53 alexmiller updated, new sha d0b33e0d346736aa985c150145b332f97b92135e #2019-06-30 05:26:34 seancorfield Thank you! #2019-06-30 05:42:50 seancorfield My dot-clojure repo is updated to use that SHA for :deps. #2019-06-30 05:43:21 seancorfield Not sure when I'll do the add-lib screencast. I'll probably try to find time this week. #2019-06-30 21:23:44 geraldodev This "Bill of materials" from maven is not existent on tools-deps because it already compute the transitive dependencies rightly ? Is it like a in advanced package-lock.json ? #2019-06-30 21:29:49 seancorfield @geraldodev Not quite sure what you're asking there? #2019-06-30 21:33:08 seancorfield With Maven, each library lists only what it depends on directly -- and then Maven figures out all the transitive dependencies... #2019-06-30 21:34:35 geraldodev I'm trying camunda-bpm and it has a bill of materials artifact #2019-06-30 21:36:16 geraldodev on mvnrepository it advises to import into lein like this [org.camunda.bpm/camunda-bom "7.11.0" :extension "pom"] #2019-06-30 21:36:18 seancorfield Just read that, still no idea what it is that's any different from specifying dependencies... #2019-06-30 21:38:36 seancorfield deps.edn supports :extension #2019-06-30 21:39:04 seancorfield {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "601305010e03202425332b342f304d313552352a512e"}, :content ("[email protected]")} is that what you're looking for? #2019-06-30 21:40:45 geraldodev The right way of using this camunda-bom #2019-06-30 21:42:53 seancorfield I thought there was a syntax involving $ for classifier support but this seems something different (simpler). #2019-06-30 21:45:16 geraldodev what :extension "pom" does ? #2019-06-30 21:46:44 seancorfield I don't know much about Maven but my understanding is that :extension "pom" tells the system to look for a .pom file instead of a .jar file -- which is what you need for BOM, right? #2019-06-30 21:48:04 geraldodev From what I understood this bom is a dependency that its like a package-lock.json and later the artifact is enlisted as a dependency minus its version, and maven uses this blessed list to fetch the right version #2019-06-30 21:49:00 seancorfield I've no idea what a package-lock.json file is, BTW. #2019-06-30 21:49:59 geraldodev if you clj -Stree '{:deps {org.camunda.bpm/camunda-bom {:mvn/version "7.11.0" :extension "pom"}}}'|grep camunda camunda its not listed, so I assume it's not being seen like a dependency #2019-06-30 21:51:32 geraldodev @seancorfield thx helping me. I'll no use the BOM artifact. #2019-06-30 21:52:35 seancorfield Yeah, so it looks like you'd have to specify all the camunda deps explicitly in deps.edn if pulling in via the bom doesn't work... #2019-06-30 21:54:36 seancorfield @alexmiller can probably explain more of the differences between how lein/`boot` work and how tools-deps works around that. #2019-06-30 21:57:25 seancorfield Ah, interesting. Maven is a giant mystery to me and I try hard to avoid it as much as possible 🙂 #2019-06-30 22:01:43 geraldodev me too, but I'm trying to use camunda-bpm-engine on a tools-deps project #2019-07-01 13:47:07 dharrigan This may be a daft question, but if I switch JVMs (from 12 to 8 ) on the command line, and I restart the repl, do I need to clean up anything to ensure that JVM 12 bytecode is gone? Or is it all in memory etc...? #2019-07-01 13:47:27 dharrigan I'm trying to track down a really really difficult to diagnose interop issue and I'm starting to run out of options. #2019-07-01 13:48:34 ghadi same as lein #2019-07-01 13:48:42 ghadi it's all in memory unless you've AOTed something to disk #2019-07-01 13:49:00 ghadi you should post more specifics about the interop issue @dharrigan #2019-07-01 13:50:12 dharrigan I've posted the problem in #interop, no response yet. #2019-07-01 13:50:13 alexmiller bytecode is only created if you 1) AOT (and you typically tell it where to go, like target/classes) or 2) in memory when loading source. It's not cached anywhere else. #2019-07-01 13:52:46 dharrigan basically (and I don't really want to get into a crosspost/in-depth here), a one-liner in Kotlin message.toByteArray() where message = protobuf, fails miserably with clojure (.toByteArray message), with clojure complaining about illegalaccesserror. So, I even tryed to extract that code into a little kotlin module by itself and invoke kolin code from clojure (that does the same thing) but I see the same issue. It feels like the JVM isn't finding all the classes <shrug> no idea. #2019-07-01 13:53:36 alexmiller prob better to have this discussion in #interop #2019-07-01 13:53:58 dharrigan I see your message 🙂 #2019-07-01 16:51:56 Travis with tools deps is there a way to add built java classes to the classpath ? For instance I have a project using GRPC which is protobuf java classes that are generated. #2019-07-01 16:52:50 ghadi :paths #2019-07-01 16:53:12 ghadi ^ corresponds to classpath directly #2019-07-01 16:53:25 ghadi or :extra-paths if you're in an alias #2019-07-01 16:53:49 Travis ok, so just add a path to where the classes exist #2019-07-01 16:53:54 ghadi yup #2019-07-01 16:54:03 ghadi to the output directory of protobuf, probably #2019-07-01 16:54:04 Travis gotcha, wasn’t sure if that worked for class files #2019-07-01 16:54:35 ghadi for $dir/foo/bar/Something.class
you'd add $dir #2019-07-01 16:54:49 Travis perfect #2019-07-01 19:34:16 dnolen hrm in order to resolve conflicts between git deps & mvn deps (in this case ClojureScript) it should be possible to use :exclusions right? Has anyone else got this working? #2019-07-01 19:34:44 dnolen I keep seeing: Error building classpath. Unable to compare versions for org.clojure/clojurescript: {:mvn/version "1.10.516", :deps/manifest :mvn} and {:git/url " #2019-07-01 19:35:16 dnolen Odd because -Stree doesn't show org.clojure/clojurescript after my exclusions #2019-07-01 19:55:12 alexmiller @dnolen not sure if that msg connects to something above? #2019-07-01 19:55:59 dnolen to clarify, I got the above error #2019-07-01 19:56:10 dnolen then I ran clj -Stree and I saw ClojureScript coming from maven #2019-07-01 19:56:39 dnolen I used -Stree to systematically remove ClojureScript maven artifacts via :exclusions #2019-07-01 19:57:16 dnolen still get the same error #2019-07-01 20:02:23 dnolen also I don't see ClojureScript except for the git one when I run clj -Spath #2019-07-01 20:13:05 alexmiller "got the above error" refers to what? #2019-07-01 20:19:42 dnolen Error building classpath. Unable to compare versions for org.clojure/clojurescript: {:mvn/version "1.10.516", :deps/manifest :mvn} and {:git/url " #2019-07-01 20:19:47 dnolen @alexmiller ^ #2019-07-01 20:22:10 dnolen (was that missing before?) #2019-07-01 20:51:32 alexmiller yes :) #2019-07-01 20:51:48 alexmiller now I see it, maybe slack being weird #2019-07-01 20:52:44 alexmiller the resolution algorithm may download things that don't show up in the final tree as it goes through the process of walking the deps and making decisions #2019-07-01 20:53:08 dnolen I guess I'm confused as it to it barfs #2019-07-01 20:53:10 dnolen and doesn't start #2019-07-01 20:53:21 dnolen oh but maybe something adds it? #2019-07-01 20:53:35 dnolen though I guess there's no way for me to see what? #2019-07-01 20:53:37 alexmiller well at some point it's encountering the maven version and then trying to decide whether it's newer/older than the git version #2019-07-01 20:53:44 alexmiller you can add a hidden verbose flag #2019-07-01 20:54:29 alexmiller clj -Sdeps '{:aliases {:v {:verbose true}}}' -A:v #2019-07-01 20:55:51 alexmiller not really designed to be understandable to anyone but me. I'm not sure you'll get the parent child linkage to know where it's coming in from that #2019-07-01 20:56:26 alexmiller if you specify the clojurescript as a top dep, it should never need to make a decision though, so I'd expect that to work regardless #2019-07-01 20:57:29 alexmiller if you want me to look at it in more depth, send me the full deps.edn and the verbose output if you have it, don't need to spam the channel here with it #2019-07-01 20:58:33 dnolen let me see if I can just make this minimal #2019-07-01 20:58:44 dnolen will drop something later, have to head for now #2019-07-01 21:03:45 alexmiller np #2019-07-02 08:53:26 ikitommi is there an example out there how to do mixed java + clj projects with deps? #2019-07-02 08:57:36 lispyclouds @ikitommi AFAIK tool.deps doesn't support java compilation. Only way I know to get around it is build the java project separately and import it as a dep in deps.edn. #2019-07-02 08:59:17 ikitommi thanks, but that was not the answer I was hoping for. #2019-07-02 09:00:36 Olical I messed around with JNI stuff and deps.edn but basically did the compiling myself through a Makefile, probably not helpful but still. https://github.com/Olical/snowball #2019-07-02 09:00:40 lispyclouds yeah I had asked about it a while back and got the answer. compiling java is beyond tools.deps scope i think and it makes sense #2019-07-02 12:32:29 alexmiller My two suggestions are either to use an additional build tools around clj or to wall off your Java bits in their own project with their own artifact build #2019-07-02 13:15:03 seancorfield @ikitommi you could always cheat and use the -e option to run clojure.java.sh and shell out to the Java compiler 😇😂:zany_face: #2019-07-02 13:18:21 ikitommi great idea Sean 😉 did a external compile for my case, but puzzled that has the idea been that people would start using deps instead of boot or lein? Still need a “do-it-yourself” build tool on top - not the most welcoming newbie experience. Some official lein’ish battery-pack would be great? #2019-07-02 13:19:11 ghadi newbies don't want to compile java 🙂 #2019-07-02 13:19:26 ghadi but battery-pack is something we've talked about #2019-07-02 15:31:23 ewen badigeon can indeed compile java, let me know if you have any issue with it #2019-07-02 17:28:33 hiredman someone in #core-async was having issues using badigeon to build an uberjar and the errors they were getting made me thing badigeon was double aot compiling core.async somehow, but looking at it again, I suspect badigeon may have an issue with timestamps so the clojure source files can end up being tagged with a newer time then the aot'ed code, which can break things #2019-07-05 11:20:42 ewen @U0NCTKEV8 yes thanks you were right. I forgot to set the last modification time when moving files to a jar/zip archive #2019-07-02 15:34:02 Travis I am about to try out badigeon myself for compiling some protobuf java classes #2019-07-02 15:49:54 seancorfield @ikitommi Given how many beginners I've seen get tripped up with ~/.lein/profiles.clj in the last few months, I think there's a good argument for simpler tooling like the Clojure CLI. I've seen beginners struggle a lot less with the CLI than you might think. #2019-07-02 15:50:30 seancorfield And, yes, as @ghadi says, "newbies" aren't trying to compile Java and Clojure together. #2019-07-02 15:51:43 seancorfield At work, we do everything with the CLI and a small handful of (sharp) tools. We have a shell script purely to automate repeated sequences of clojure invocations. #2019-07-02 16:10:41 dominicm @ikitommi in pack.alpha I do some clever stuff to compile java using the JDK api. Not much better than shelling out 🙂 #2019-07-02 16:11:03 dominicm I have toyed around with the idea of using the ztellman library for reloadable java with tools.deps but never actually done it. #2019-07-02 17:31:21 dnolen @alexmiller circling back - the problem goes away if I supply a ClojureScript git dep version in the root project. Is the behavior I observed yesterday a bug, or just the current behavior? if I supply all the exclusions should I be able to avoid the warning about comparing versions? #2019-07-02 17:45:13 alexmiller not sure #2019-07-02 17:45:47 alexmiller walking the full deps tree necessarily requires comparing versions of things, and comparing mvn to git is not (yet) supported (although I know how to do it) #2019-07-02 17:46:40 alexmiller the question is whether in this particular scenario, with exclusions, the comparison can be avoided. exclusions are tricky and it's hard for me to answer that question without having the full scenario in hand. #2019-07-02 17:46:50 dnolen k #2019-07-02 18:10:36 dnolen @alexmiller ok when I made a minimal case :exclusions worked for me - so I guess -Stree can't be used reliably? #2019-07-02 18:28:00 dnolen hrm I guess perhaps a comparison happens before you know it will be excluded? #2019-07-02 18:29:42 alexmiller yes, and as you encounter more things, it's possible for the excluded state of something to change #2019-07-02 18:29:55 dnolen verbose output just seems to confirm that my exclusions worked #2019-07-02 18:30:17 dnolen but the end behavior is surprising #2019-07-02 18:30:18 alexmiller -Stree tells you the end state, but is not the whole story of how it got there #2019-07-02 18:30:28 dnolen since for the end user this doesn't seem deterministic #2019-07-02 18:30:32 dnolen sometimes exclusions work #2019-07-02 18:30:34 dnolen sometimes it doesn't #2019-07-02 18:31:03 dnolen it also breaks using git deps transitively #2019-07-02 18:31:04 alexmiller yes, there is a ticket in this area (specifically around cljs iirc, although not with a git dep) #2019-07-02 18:31:10 dnolen it can't be done safely #2019-07-02 18:31:26 dnolen so you have to re-declare your deps at the bottom #2019-07-02 18:31:41 alexmiller I think you are jumping to conclusions that are not necessarily true #2019-07-02 18:32:06 dnolen so you have some way to avoid the above? #2019-07-02 18:32:32 dnolen I have to re-declare my git deps right now #2019-07-02 18:32:57 alexmiller it is possible to declare deps trees that are either logically inconsistent (impossible to satisfy the deps and exclusions as stated across the tree) or that are ambiguous #2019-07-02 18:33:17 dnolen but that's not what I've described above #2019-07-02 18:33:32 alexmiller well you haven't given me a repro, so I can say either way #2019-07-02 18:34:06 dnolen yeah I don't know how make this one since it's non-trivial project with non-trivial deps #2019-07-02 18:34:12 dnolen it works in the trivial case #2019-07-02 18:34:19 alexmiller most things do :) #2019-07-02 18:35:00 dnolen ... oh #2019-07-02 18:35:15 dnolen other git stuff won't appear in -Stree? #2019-07-02 18:35:50 alexmiller -Stree should be the full tree of everything chosen at the end (it's just the inverted lib map which you can find in the .cpcache in the .lib file) #2019-07-02 18:36:20 alexmiller so other git stuff will appear there #2019-07-02 18:36:53 alexmiller it does not list things that were children but not chosen (due to exclusions or version selection or being dominated by a top dep) #2019-07-02 18:37:06 dnolen ah #2019-07-02 18:37:14 alexmiller sorry, I have to run to the dentist #2019-07-02 18:37:22 dnolen one second maybe I see a way to repro now #2019-07-02 18:37:29 dnolen k, will drop a link to something minimal if I can make it #2019-07-02 20:19:44 dnolen I found the issue which arose from a misunderstanding on my part #2019-07-02 20:20:47 dnolen I had a project which delcared a ClojureScript dep - and of course there were transitive deps that also declared ClojureScript but no conflicts because I guess clj considers the deps.edn :deps to be overrides? #2019-07-02 20:21:19 dnolen once I was using that project transitively - those became conflicts #2019-07-02 20:22:35 dnolen I had removed the conflicts from the downstream project, but not this upstream one #2019-07-02 20:39:27 alexmiller On your question above about overrides, I’m not sure what you mean but I suspect you’re making up a model there. Top level deps are always taken as overriding decisions of the transitive deps #2019-07-02 20:40:08 dnolen that's what I meant #2019-07-02 20:40:14 alexmiller When you move to your “upstream” project though, the deps of the “downstream” project are no longer top deps #2019-07-02 20:40:18 dnolen "top-level" #2019-07-02 20:40:41 alexmiller So are subject to other selections #2019-07-02 20:40:48 dnolen yeah that was the model I came up w/ based on what I saw #2019-07-02 20:40:52 dnolen so good to hear that confirmed #2019-07-03 05:51:20 ikitommi @dominicm thanks, alias / sample to pre-compile java classes would be good for many cases. Also enjoying Zach’s hot reload lib (https://github.com/ztellman/virgil) with lein. The latest version has never worked for me thou. Many of our newer libs include Java, because of perf, client projects because of… legacy. #2019-07-03 05:52:01 ikitommi Agree that it’s not a most common thing, but still important for the interop story. #2019-07-03 05:59:21 ikitommi What about the classpath with shell scripting the java compilation? The Java classes might need the project dependencies, including Clojure itself. Resolved that manually just for this case, but would appreciate a sample how to combine full library classpath from deps to feed into a custom javac step. Will check pack.alpha too. #2019-07-03 12:15:47 alexmiller clj -Spath ? #2019-07-03 08:34:36 dominicm @ikitommi just had an internal discussion about java compilation, the topic of windows came up as problematic for shelling out. If I was investigating this today, I would probably lean on the JDK API again because of that. #2019-07-05 03:18:29 seancorfield This supports multiple templates in a single git repo and makes it easier to have conditional files in a template (`->files` accepts and ignores nil entries). #2019-07-05 08:05:14 dominicm On a similar vein, @viesti just added support to https://github.com/juxt/pack.alpha for building docker images straight from Clojure! With a big thanks to him for that! #2019-07-05 08:05:36 dominicm (Now, I need to go and incorporate the clj-new changes into Edge!) #2019-07-05 14:20:55 alexmiller clj 1.10.1.462 was released yesterday - no changes in tools.deps, but this undid some of the installer changes so that the install deps.edn is being installed again. this allows older tools.deps lib users relying on the installed clj's reporting of config files via clj -Sdescribe to continue working without update. We saw some issues with this in Datomic ion deploy and Cursive. #2019-07-05 21:05:23 cfleming @alexmiller Sorry, I didn’t get time to look at that yesterday. #2019-07-05 21:06:00 cfleming I’ll try to get to it soon - it’s the weekend here and I’ve been busy dealing with IntelliJ bugs unfortunately. #2019-07-05 22:13:15 cfleming Deps doesn’t support global exclusions, right? i.e. I don’t want a particular dep anywhere. #2019-07-05 23:05:25 alexmiller Not yet, no #2019-07-05 22:29:59 seancorfield I've just identified a potential security flaw in seancorfield/depstar 0.2.1/0.2.2/0.2.3 and I have released 0.2.4 to address it. If you are using those earlier versions, please update to the new release a.s.a.p. #2019-07-06 03:43:23 dominicm @seancorfield what's the security issue? #2019-07-06 04:09:05 seancorfield @dominicm The PR I accepted for 0.2.1 allows for depstar to tree walk through the file system and overwrite files outside the temporary directory. #2019-07-06 04:11:05 seancorfield I didn't realize until it failed to build an uberjar on our CI system -- which had always worked before, strangely -- but failed once it tried to create the parent directory (new in 0.2.1). #2019-07-06 04:13:19 seancorfield (so I'm not entirely certain the bug was introduced in 0.2.1 but that was the build that first caused "random" directories to be created if you tree-walked, which wasn't good) #2019-07-06 04:13:45 alexmiller and yet you want Clojure to create classes directories... ;) #2019-07-06 04:14:40 seancorfield The security issue isn't technically the auto-creation of the directory... #2019-07-06 04:15:19 seancorfield ...I suspect the bug's been there for quite a few releases prior but the behavior was masked until the mkdirs on the parent folder was added. #2019-07-06 04:18:52 seancorfield (and the behavior was somewhat benign before 0.2.1) #2019-07-06 13:12:18 geraldodev #2019-07-06 13:12:32 geraldodev I'm using reply (the one that comes with lein) and it initiates a nrepl server . Problem was the it starts nrepl in a way that it does not write .nrepl-port file that my beloved Fireplace (vim) needs to connect to nrepl. In case you have the same problem I'm asking reply at init to do this by using the snippet attached #2019-07-06 13:24:48 geraldodev @seancorfield I'm using your dotclojure file. the reply section was made by you. I've checked today and you do not have reply anymore. What repl client are you using ? #2019-07-06 16:36:15 seancorfield I don't remember ever having reply in there @geraldodev but it does have both the old and new nREPL aliases. I use a Socket REPL and Chlorine for Atom for editing or unravel/unrepl from the command line. #2019-07-06 18:40:50 dominicm @geraldodev do you use the reply in the terminal? If not I would recommend using nrepl.cmdline, which does write the .nrepl-port file (and deletes it afterwards). #2019-07-06 20:45:59 geraldodev @dominicm I use reply on terminal. Since I've looked up reply on sean's dot-clojure I've seen this nrepl.cmline there and I've invoked to see if it would be less complicated than the eval parameter that I'm passing to reply. But no, it writes .nrepl-port but I would have to invoke reply (the client) as another proccess which I was doing before. Invoking reply directly it 1) starts nrepl 2) writes .nrepl-port and Fireplace can consume this. I get one less java process. I dealt with this problem because I'm working with 2 repls . One for server and another (swing) for client. That would be 4 process and one more tmux screen. #2019-07-06 21:16:50 dominicm I'm a fellow fireplace user, I understand, mostly! I don't find myself reaching for the terminal repl at all. #2019-07-07 13:08:18 dharrigan Was there discussion around deps.edn supporting alias(es) of aliases, or does that go too far? #2019-07-08 05:56:39 lepistane i think it already does #2019-07-08 06:48:18 dharrigan I'm not seeing that described in the deps_and_cli documentation #2019-07-08 06:48:26 dharrigan (in the aliases section) #2019-07-08 06:52:02 yogidevbear Then again ☝️ #2019-07-08 06:52:07 dharrigan Yes, I read that #2019-07-08 06:52:21 dharrigan but nowhere does it say, or imply or give example that an alias can reference another alias #2019-07-08 06:52:23 dharrigan i.e., #2019-07-08 06:52:35 yogidevbear Oh I'm with you now #2019-07-08 06:53:01 dharrigan {:aliases {:foo ....
:bar :foo} #2019-07-08 06:53:03 dharrigan or something like that #2019-07-08 06:53:11 dharrigan where bar can pull in the configuration of foo #2019-07-08 06:53:26 dharrigan so intead of having a big command line like #2019-07-08 06:54:11 dharrigan clj -A:repl:bar:test:foo you can do cli -A:super-aliases-to-the-max that is the aggregation of repl:bar:test:foo #2019-07-08 06:55:08 yogidevbear Lol, love the choice of naming -A:super-aliases-to-the-max #2019-07-08 06:55:15 dharrigan 🙂 #2019-07-08 07:04:28 yogidevbear I might be reading the source incorrectly, but looking through the t.d.a reader.clj, I don't think alias of aliases is supported. Disclaimer: insert /giphy I don't know what I'm doing 😉 #2019-07-08 08:14:58 dominicm Inheritance is not supported. I don't think it's going to be, but I'm not qualified to make the assertion on the latest thinking. #2019-07-08 08:25:42 sogaiu after a while, generating one's deps.edn from something else starts to seem attractive 🙂 #2019-07-08 09:06:19 kirill.salykin hi,
I am trying to add athena-jdbc jar as local/root
but without any success
[ERROR] 'artifactId' with value 'AthenaJDBC${env.JDBC_V}' does not match a valid id pattern. @
#2019-07-08 09:06:37 kirill.salykin Is this something wrong with pom file? #2019-07-08 09:07:00 kirill.salykin <groupId>Athena</groupId>
<artifactId>AthenaJDBC${env.JDBC_V}</artifactId>
<version>${env.MAJOR_V}.${env.MINOR_V}.${env.REVISION_V}.${env.BUILD_V}</version>
<packaging>jar</packaging>
<name>AthenaJDBC${env.JDBC_V}</name>
<url>http://maven.apache.org</url>
#2019-07-08 09:07:25 kirill.salykin please advice #2019-07-08 09:33:38 dharrigan @dominicm I wonder why not supported? I'm sure it was thought about/discussed/hammocked. #2019-07-08 20:01:15 felipebarros For that use case, why not just use the alias functionality of your shell? It may not be ideal but it will surely do what you want. max-alias RET. #2019-07-12 10:56:02 dharrigan True true #2019-07-08 17:42:48 seancorfield @kirill.salykin That pom.xml file is expecting you to have environment variables set to specify all those variables. If you have that as a local dep, I suspect that the project is expecting the final pom to be built as part of the build/deploy process -- and does not expect you to try to use the project from source. #2019-07-08 18:35:39 kirill.salykin Thanks! You are the mvp, was a bit surprised that jar contains half baked pom #2019-07-09 15:04:09 y.khmelevskii Hi gents! Can you please help me to understand how I can reload duct server when I change required library
I use deps.edn together with lein
{:paths ["src"]
:deps
{...
shared {:local/root "../shared"}}
and when I change files in shared dependency I can’t apply them #2019-07-09 15:05:01 kirill.salykin I might be mistaken, but I think you should restart the repl #2019-07-09 15:05:29 kirill.salykin so deps.edn can rebuild classpath #2019-07-09 15:05:54 kirill.salykin there was something which supports dynamic load
add-lib I think, but dunno much about it #2019-07-09 16:08:39 y.khmelevskii Hmm, it’s not good idea to restart repl every time when I changed shared dependency #2019-07-09 16:15:01 seancorfield There's nothing different about using deps.edn vs lein in terms of reloading your app while you're working. I do it all the time with our Component-based apps: stop the system, reload any namespaces you need to, start the system again. #2019-07-09 16:17:01 seancorfield (although, to be honest, I almost never need to do that as I eval code into the running image all the time while I'm working, and I use Var references for a few key things around routes and middleware so I don't need to restart the app) #2019-07-09 16:20:27 kszabo @y.khmelevskii in case of local dependencies, there’s no need to restart the jvm, since the dep folders are on your classpath you can just use cider-ns-reload-all or your editor’s equivalent to get the latest state #2019-07-09 16:22:32 kirill.salykin Oh, I misunderstand you
If it is already on the class path - you just need reload tell, not restart #2019-07-09 16:26:10 kenny A bit of a tangent... it'd be useful to be able to switch from a git/maven dependency to a local one at runtime. That would remove my primary reason for needing to restart the REPL. #2019-07-09 16:59:18 alexmiller That’s not a thing that would be possible to do in general for stuff that’s already loaded #2019-07-09 19:39:16 dominicm It's not so bad when combined with t.n.s #2019-07-09 19:39:22 dominicm You can run a refresh afterwards. #2019-07-09 16:27:29 ghadi not possible #2019-07-09 16:29:25 kenny For all dep type cases or just maven? #2019-07-09 16:32:10 seancorfield I strongly recommend developing a tight REPL-Driven Development workflow where you eval code as you change it -- small, fast feedback loop -- rather than making a bunch of changes and then trying to restart/reload your app. Then these sorts of questions just don't even come up. I start my REPL/REBL combo up and tend to leave it running for days, sometimes even a week or more, and I might only need to restart my app once or twice a week. #2019-07-09 16:33:38 kenny @seancorfield Not sure how that would solve this problem. The dependencies are immutable so you cannot make changes to them. #2019-07-09 16:34:36 seancorfield How often do you find yourself changing dependencies? (you can add new dependencies on the fly via the add-lib branch of tools.deps) #2019-07-09 16:36:06 seancorfield We have an 80K+ line code base at work and dozens and dozens of external lib dependencies -- and we update them in a controlled manner every few weeks or so. #2019-07-09 16:36:15 kenny Often. Currently we have ~15 internal libraries that a particular service may depend on. They are all git deps. If you find a bug or need to add something to a particular library, you need to stop your repl and then decide to either change the library dep to a local/root or push a new version of the lib. #2019-07-09 16:37:09 kenny It's not usually adding a dependency - that's rare. It's almost always updating to a newer version or switching to local/root. #2019-07-09 16:41:33 seancorfield Interesting. We don't run into that because we have a monorepo -- so all code is already a local dep. We can edit any of it or pull new versions down and, at worst, just reload all on the main namespace of the subproject we're working on. #2019-07-09 16:42:34 seancorfield (one of the main drivers for our switch from Boot to the CLI tools was that it would work better with our monorepo -- because we were having problems with Boot's pods and the shadow fileset thing it does) #2019-07-09 16:45:02 kenny Yeah I have been seriously considering switching us to a monorepo to remove this pain point. I hate having to restart the REPL to go and fix a small bug. #2019-07-09 16:46:43 kenny I've also thought about some sort of hackery that would temporarily update the .gitlibs file to a local/root file so you could dynamically switch between a git dep and a local one. Haven't written anything around that concept yet though. It feels a bit icky but so is restarting the REPL a lot. #2019-07-09 16:48:10 mccraigmccraig @kenny we used to have multiple repos and switched to a monorepo - so much better #2019-07-09 16:49:35 kenny @mccraigmccraig @seancorfield How do you guys deal with ensuring tests/build steps for a particular project only run when that project changes? #2019-07-09 16:50:13 mccraigmccraig @kenny we don't . we run CI across all modules in the monorepo on every commit #2019-07-09 16:50:49 kenny Ah. That seems tricky. That could easily result in extremely long build times for simple changes. #2019-07-09 16:53:09 mccraigmccraig sure, a CI run is 10 mins for us, but it doesn't matter - your local dev loop is generally to run tests in one namespace from the editor and is lightning fast #2019-07-09 16:54:58 kenny 10 mins would be ok but I don't think we'd be close to that. Many services need to build docker images, run integration tests, build cljs, and run hundreds of thousands of gen tests. I could see the build time easily going over an hour. #2019-07-09 16:57:16 mccraigmccraig you could look at the paths changed by the last commit and only run tests in changed modules #2019-07-09 16:58:41 mccraigmccraig although presumably you would also want a full run sometimes #2019-07-09 17:01:29 kenny Something like that may work. Though there doesn't appear to be any tooling around this sort of stuff yet. #2019-07-09 17:04:10 kenny I've been very close to digging in on that front because of how painful the REPL restarts are. A monorepo seems like quite a useful tool for development but it also feels like it may open a different can of worms. Blog posts around this topic would be very useful 🙂 #2019-07-09 17:25:02 seancorfield @kenny (sorry, stepped away) I mostly run tests directly from the editor while I'm developing but we can run tests selectively for any combination of subprojects locally from the command line or a full suite end-to-end for a CI-style sweep. Again, tho', working in a tight feedback loop -- eval'ing code as you write it and either running test expressions from Rich Comment Forms or actual tests via the editor -- means that your risk of breaking anything is minimal. #2019-07-09 17:40:41 kenny @seancorfield Right - the dev story is great. I was talking about CI time for a monorepo and how a single push could easily result in very long build times. #2019-07-09 18:26:55 ghadi @kenny while developing locally on 15 dependent libs, I'd definitely use local/root #2019-07-09 18:27:11 ghadi Exclusively #2019-07-09 18:28:08 ghadi Because of the bug fix scenario you described #2019-07-09 18:28:34 kenny You'd then need to guarantee that all the devs on your team have the same directory structure, I'd think. #2019-07-09 18:28:55 ghadi No, just generate the deps file #2019-07-09 18:29:02 ghadi It's data for a reason #2019-07-09 18:29:45 kenny Not sure what you mean. Also, I think that'd lose the valuable comments that we have in our deps.edn. #2019-07-09 18:31:44 ghadi I mean: programmatically output the local deps file with some source file #2019-07-09 18:31:53 ghadi Can comment the source file #2019-07-09 18:32:09 kenny Oh, interesting #2019-07-09 18:32:16 seancorfield We auto-generate an everything/deps.edn file from all the other deps.edn files (each subproject has its own) and then we generally run a REPL/REBL based on that "everything" deps file. #2019-07-09 18:32:23 ghadi ^^ #2019-07-09 18:33:01 ghadi If you use git deps locally, that conflicts with manual iteration on the dependency #2019-07-09 18:33:27 ghadi A git SHA is a value, local dir is a mutable zone #2019-07-09 18:33:34 kenny So you have a deps-base.edn (or something like that) and then when you spin up a clj process, you generate a deps.edn based on the "environment" you want? #2019-07-09 18:33:41 ghadi Yeah #2019-07-09 18:33:58 ghadi Or generate it one time for a developer onboarding #2019-07-09 18:34:07 ghadi Or whenever the source file changes #2019-07-09 18:34:22 ghadi Exactly #2019-07-09 18:35:49 kenny That seems doable and would remove the need for a monorepo. This seems like very valuable info that should be a part of the deps.edn doc if it's not already. #2019-07-09 18:37:29 ghadi monorepo is a big hammer #2019-07-09 18:38:18 ghadi just a little read => update => spit is enough to avoid that hammer #2019-07-09 18:41:01 seancorfield > This seems like very valuable info that should be a part of the deps.edn doc if it's not already.
I think different people have different approaches -- I don't think you can generalize this sort of thing for the base set of documentation. #2019-07-09 18:42:24 kenny True. An outline of various approaches you could use to solve these problems in the real world would be fitting. #2019-07-11 08:18:36 borkdude lately I'm getting
Full report at:
/var/folders/2m/h3cvrr1x4296p315vbk7m32c0000gp/T/clojure-6407140529537279679.edn
when I get an exception running a program with tools.deps. Is there a way to get the old behavior back? #2019-07-11 08:22:24 kirill.salykin --report=stderr
I think #2019-07-11 08:22:37 kirill.salykin @borkdude #2019-07-11 08:24:12 borkdude > file - write to a temp file (default, falls back to stderr)
when would this "fall back to stderr" as a default? #2019-07-11 08:24:26 borkdude if it has no write access? #2019-07-11 08:29:12 borkdude This doesn't work for me:
clj --report stderr -A:clj-kondo --lint corpus/unresolved_symbol.clj
#2019-07-11 08:29:30 borkdude It says "-A:clj-kondo (No such file or directory)" #2019-07-11 08:33:09 kirill.salykin maybe try with -A:clj-kondo in the end? #2019-07-11 08:33:33 borkdude --lint ... are arguments to clj-kondo #2019-07-11 08:38:16 borkdude it worked with the java property #2019-07-11 08:45:27 vlaaad --report stderr are clojure.main arguments right? #2019-07-11 08:46:24 vlaaad maybe you need to modify :clj-kondo alias's :main-opts and put --report stderr there? #2019-07-11 08:46:42 borkdude yeah, that makes sense #2019-07-12 10:17:41 dominicm I think that a JVM option variant was added too, for leiningen. #2019-07-12 10:18:05 borkdude I'm using that right now, works #2019-07-12 10:20:41 dominicm > Specifically, the default behavior for error reporting with Clojure 1.10.1-beta3 is to write the error report and stack trace to a file, but you can also change that to stderr via system property -Dclojure.main.report=stderr if you prefer. #2019-07-12 10:21:04 borkdude yes, that one #2019-07-11 08:46:56 kirill.salykin seems reasonable #2019-07-11 14:31:37 kirill.salykin If I copy jar to .m2 and add it to deps.edn - it works
mkdir -p ~/.m2/repository/athena/athena-jdbc/2.0.7/
wget -O ~/.m2/repository/athena/athena-jdbc/2.0.7/athena-jdbc-2.0.7.jar
:deps {athena/athena-jdbc {:mvn/version "2.0.7"}
but if I add it via local/root - tools.deps starts to complain about wrong data in pom
:deps {athena/athena-jdbc {:local/root “lib/athena-jdbc.jar”}
why is such difference? is there a way to add jar as from .m2 but from within the project? #2019-07-11 14:49:19 ghadi if you get an error, can you post it in a snippet (COMMAND-SHIFT-ENTER) @kirill.salykin #2019-07-11 14:49:49 kirill.salykin yes, one moment #2019-07-11 14:49:51 ghadi I would like to see the error from when you add it via :local/root #2019-07-11 14:53:08 kirill.salykin #2019-07-11 14:53:12 kirill.salykin @ghadi ^ #2019-07-11 14:53:40 ghadi ok this is what I suspected #2019-07-11 14:54:23 ghadi when tools.deps uses a local jar with :local/root && the jar contains an internal pom.xml, it will try to read the pom and fetch dependencies #2019-07-11 14:54:37 kirill.salykin shall I remove all poms? #2019-07-11 14:54:51 ghadi yeah you could #2019-07-11 14:55:04 kirill.salykin and why it doesnt try to do it when reading from .m2? #2019-07-11 14:55:04 ghadi only one pom will matter, I forget the exact path #2019-07-11 14:55:37 ghadi it will try to read the pom in .m2 - but not the pom within the jar, the one next to the jar if present #2019-07-11 14:55:46 ghadi Maybe ^, I forget the exact details #2019-07-11 14:55:56 kirill.salykin there is no pom with the jar #2019-07-11 14:56:10 ghadi But anyways, if it goes through the .m2 path that's a different handler entirely -- uses Maven Resolver #2019-07-11 14:56:15 kirill.salykin only this published by aws
#2019-07-11 14:56:34 ghadi kill the pom from the jar and you can use it in local/root #2019-07-11 14:56:38 kirill.salykin thanks #2019-07-11 14:57:13 kirill.salykin does it make sense to add some flag to force tools.deps not try to fetch deps defined in pom?
what do you think? #2019-07-11 14:57:56 kirill.salykin eg
{:deps {jdbc {:local/root "...", :ignore-pom? true}}} #2019-07-11 14:58:36 ghadi I'll have to check with Alex about why it does that in the first place #2019-07-11 14:58:44 ghadi before making solutions #2019-07-11 14:58:58 kirill.salykin of course, just a little proposal #2019-07-11 14:59:02 kirill.salykin thanks a lot for help #2019-07-11 14:59:04 ghadi np #2019-07-11 14:59:52 ghadi 😂 been pasting that everywhere ^ #2019-07-11 15:03:05 kirill.salykin ) #2019-07-11 15:42:21 seancorfield Can you specify the JAR in :paths and not worry about dependency resolution? #2019-07-11 15:48:05 ghadi ^^^ oh of course #2019-07-11 15:48:13 ghadi @kirill.salykin #2019-07-11 16:05:27 kirill.salykin Oh, I ll try, thanks! #2019-07-11 16:12:04 alexmiller If you’re using a jar, that has transitive dependencies, how else are you going to satisfy those deps? #2019-07-11 16:13:56 ghadi most of them don't have transitive deps @alexmiller -- the usecase is usually downloading some database driver that is self-contained in the jar #2019-07-11 16:14:25 ghadi if you were using bare java, you'd put it on the classpath directly #2019-07-11 16:15:38 ghadi which means, use :paths #2019-07-11 16:16:51 alexmiller yes, tools.deps purpose is to build out transitive deps, and it will do so with local jars as well. The no deps case is kind of the base case and it doesn’t matter #2019-07-11 16:41:33 kirill.salykin Thanks for clarification #2019-07-12 16:05:35 kenny @ghadi We talked on Tuesday (https://clojurians.slack.com/archives/C6QH853H8/p1562697498052500) about a tool that would "localize" (i.e. git/url -> local/root) certain dependencies before starting a REPL (or some other task). This would solve the problem of having a single project that depends on lots of internal libraries that often need to be developed in parallel with the project you are working on. You said "monorepo is a big hammer" and to take a "`read` => update => spit" approach. The general approach makes sense but there are a few issues I immediately ran into while trying to work on this tool. I'm curious how you solved these for your company (it sounded like you guys have a tool that does this too).
When you "localize" dep libraries, how do you ensure that the library at a give local/root path is up to date with VCS? Often my local libraries are not up to date with the version on master. You could run a git check before localizing and warn the user that the code at a given local/root is not up to date. Similarly, should you ensure your localized dep libraries are all on the same branch? I could easily see a situation where the local/root of a lib is on a different branch (maybe you started some work there and didn't switch back to master) than the other libs or the service itself that you are working on.
Both of those issues could easily cause strange behavior to occur which could be costly when debugging because you didn't have know the lib was not up to date or on the wrong branch. Having a monorepo would solve both of these problems (it introduces another set of problems though). #2019-07-12 17:28:20 dnolen clojure -J-Xms4g -J-Xmx8g ... #2019-07-12 17:28:31 dnolen this should set the JVM memory settings right? #2019-07-12 17:29:22 dnolen I keep seeing OOMs from JGIT - is there a different way to set the memory for the tools deps itself? #2019-07-12 17:33:16 dnolen does clojure have it's own memory settings that have to be set independently? #2019-07-12 17:34:20 andy.fingerhut One way is to create an alias in a deps.edn file e.g. deep merge this into deps.edn map: {:aliases {:bigmem {:jvm-opts ["-Xms4g" "-Xmx8g"]}}}. Then run clojure -A:bigmem #2019-07-12 17:35:32 dnolen @andy.fingerhut so are you saying what I'm trying above won't work? #2019-07-12 17:37:03 dnolen note the OOMs are from clojure tool trying to fetch git deps #2019-07-12 17:37:13 dnolen no from launching any stuff I'm trying to run #2019-07-12 17:37:21 andy.fingerhut Oh, no, I believe it will work. I was responding more to the question I thought you were asking about a way to have settings for this. #2019-07-12 17:37:51 dnolen I guess I'm skeptical that 8g wouldn't be enough for JGIT #2019-07-12 17:37:59 andy.fingerhut And I probably misunderstood which JVM you wanted those options to go to, now that I read your question again. #2019-07-12 17:38:30 dnolen yes not my own process #2019-07-12 17:38:41 dnolen I want clojure to not fail fetching git deps #2019-07-12 17:40:03 dnolen oh erg #2019-07-12 17:41:10 dnolen @alexmiller it looks like clojure has a hardcoded memory setting of -Xmx256m?? #2019-07-12 17:41:36 dnolen I'm using clojure in a CI context, and it consistently fails #2019-07-12 17:42:01 andy.fingerhut Yeah, that is what I see in my /usr/local/bin/clojure shell script, too. #2019-07-12 17:55:43 alexmiller It’s hard coded for the process that builds the classpath #2019-07-12 17:56:51 alexmiller This is the first report I’ve seen of an issue with that but happy to mod #2019-07-12 17:57:41 alexmiller Curious what it’s doing when it fails. Maybe a git version compare? #2019-07-12 18:00:58 dnolen @alexmiller PMed the output #2019-07-12 18:01:05 dnolen I thought I could set this myself with JAVA_CMD #2019-07-12 18:01:18 dnolen but no good because of the hard-coding at the moment #2019-07-12 18:02:00 dnolen I hacked it by seding clojure shell script on the CI instance #2019-07-13 18:16:53 yuhan Is it possible to mix Java source files in a deps.edn project? #2019-07-13 18:17:43 andy.fingerhut I believe the answer is that you can have Java source files, but deps.edn isn't intended to do anything with them. #2019-07-13 18:18:51 yuhan so I just put them in the src/ folder and they'll get picked up by an (import) statement? #2019-07-13 18:20:11 andy.fingerhut Something (other than the clj or clojure commands) is welcome to compile Java source files to .class and/or .jar files that can be on a classpath directory specified in a deps.edn file, but having such source files around does not cause clj or clojure commands to compile them. #2019-07-13 18:20:24 andy.fingerhut You should use some other tool to cause Java source files to be compiled. #2019-07-13 18:23:13 yuhan oh I see, that clears things up - so I'll need to compile a .jar file and refer to it using a :local/root dep #2019-07-13 18:23:41 andy.fingerhut That is one way, yes. #2019-07-13 18:26:38 yuhan hmm, I guess that's what Lein is doing with its :java-source-paths key #2019-07-13 18:27:23 yuhan Just to confirm, does that work with interactive development - if I recompile the jar and refresh the clojure ns in the REPL, will the updated classes be imported? #2019-07-13 18:35:10 andy.fingerhut I have not tried, but I would guess no, with the default choice of JVM class loaders used by Clojure. Maybe there is a way using non-default classloaders to make something like that happen, but I have never done it, and I could be wrong in guessing that it is even possible. You would likely be asking for additional headaches by attempting to do so. #2019-07-13 18:39:18 andy.fingerhut But I would strongly caution you that without more expert help than me (or perhaps you have already done similar things before), you could be in for a lot of confusing debugging and frustration there. #2019-07-13 18:46:36 yuhan Thanks for the advice - I'll probably hold off on doing anything like that then, having little experience with JVM side of things #2019-07-16 16:31:02 credulous Hi. I’m a bit confused with my workflow using clj and deps.edn. Everything was so magic with lein, I never had to actually understand what was going on. 🙂 no such luxury/laziness here. #2019-07-16 16:31:36 credulous I have an app that has both an API server and a clojurescript frontend. The server is using http-kit. #2019-07-16 16:32:22 credulous In my deps.edn I have {:aliases {:run {:main-opts [ "-m" "alexandria.core" ] } } — in alexandria.core I initialize the db, start the http-kit server etc. #2019-07-16 16:32:55 credulous In my global deps.edn, I have the alias :nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.6.0"}}} #2019-07-16 16:33:44 credulous What I would like to do is start the server, and have an nrepl session start at the same time, preferably with both a repl prompt and a server to connect to from the editor (I use fireplace/vim) #2019-07-16 16:34:18 credulous Right now I’m starting the server with clj -A:run, which doesn’t start a repl as near as I can tell #2019-07-16 16:40:57 alexmiller correct #2019-07-16 16:34:49 credulous and in a different terminal clj -R:nrepl -m nrepl.cmdline, which starts a nrepl server #2019-07-16 16:41:41 alexmiller prob correct #2019-07-16 16:35:04 credulous and in a different terminal just clj to get a repl prompt #2019-07-16 16:42:16 alexmiller you might need something more here to get an nrepl client, not just a repl ? #2019-07-16 16:35:36 credulous and in a different terminal shadow-cljs watch alexandria to get clojurescript compiling plus a clojurescript repl going #2019-07-16 16:35:49 credulous I’m sure this is not the Right Way to do things. #2019-07-16 16:43:05 alexmiller you could probably write a tiny script that combines several of these things and call that instead #2019-07-16 16:43:16 alexmiller and I'm not sure if you need the clj at all? #2019-07-17 07:16:07 dominicm @credulous one problem here is that your nrepl is a different jvm to your main application. So when you eval in nrepl, it won't be reflected in your web server. #2019-07-17 14:40:49 alexmiller clj 1.10.1.466 / tools.deps.alpha 0.7.527 released:
- fix TDEPS-134 - respect Maven mirror settings in ~/.m2/settings.xml
- change error on unknown aliases to a warning
- deprecate and warn on use of clojure.tools.deps.alpha.reader/clojure-env
- change default JVM options when using tools.deps to build classpath #2019-07-17 18:53:06 dominicm I'm glad it's a warning again, I can now just spam the known common dev aliases instead of caring whether they exist 😄 #2019-07-18 21:12:10 kenny @seancorfield I updated my CI to openjdk-11 from openjdk-8 and my uberjar step using depstar has started failing with some sort of memory issue. The uberjar is built with clojure -J-Xmx3g -A:uberjar -v and the command exits with Killed and the status code 137, which typically indicates a memory issue. I'm using the latest version 0.2.4. Any thoughts on why depstar would have a memory issue on Java 11? #2019-07-18 21:14:24 kenny I am using the -v option for depstar. It's interesting that the build always hangs on this file /home/circleci/.m2/repository/http-kit/http-kit/2.3.0/http-kit-2.3.0.jar for a few seconds right before the Killed message. #2019-07-18 21:16:49 ghadi Did you set a max heap limit for depstar #2019-07-18 21:17:23 ghadi Did you look up what status 137 means? #2019-07-18 21:17:39 ghadi Ah you did set Max heap #2019-07-18 21:19:20 kenny It's strange that it's happening at all. I can't imagine it using close to 3g of mem. #2019-07-18 21:19:57 kenny It almost makes me think the -Xmx is getting ignored for some reason. #2019-07-18 21:20:42 ghadi Set it to 1GB #2019-07-18 21:21:15 kenny Same issue #2019-07-18 21:21:15 ghadi I think there was a change where depstar now unzips in memory #2019-07-18 21:21:39 kenny Ah, that sounds relevant. I'll try an older version. #2019-07-18 21:24:42 kenny It works with 0.1.7:
clojure -J-Xmx3g -Sdeps '{:deps {seancorfield/depstar {:mvn/version "0.1.7"}}}' -m hf.depstar.uberjar MyProject.jar
#2019-07-18 21:25:40 kenny It's far slower though #2019-07-18 21:27:09 kenny The code of depstar is small, should be fairly easy to find the issue. Must have something to do with Java 8->11. #2019-07-18 21:33:06 ghadi It was smaller before Sean fixed all my bugs #2019-07-18 21:39:52 kenny It's not obvious how to update to 11.0.4 for some reason. Maybe the debian packages haven't updated or something. #2019-07-18 21:44:56 ghadi dunno I use adoptopenjdk #2019-07-18 22:05:10 kenny I don't think 11.0.4 is released yet. #2019-07-18 22:10:14 kenny It's possible that issue is relevant but it's proving hard to validate given 11.0.4 isn't out yet. That issue may not even be the problem. Going to roll back to Java 8 for now. #2019-07-18 22:17:05 ghadi it's out #2019-07-18 22:17:11 ghadi adoptopenjdk has it #2019-07-18 22:17:23 ghadi @U083D6HK9 #2019-07-18 22:19:14 kenny There doesn't appear to be a debian package yet. Also couldn't find a command to force update to 11.0.4. All guides seem to suggest using an install path that doesn't let you select your version.
I don't see a 11.0.4 version on their docker page yet: https://hub.docker.com/r/adoptopenjdk/openjdk11 #2019-07-18 22:44:20 seancorfield The change with depstar was from expanding everything out to a temporary folder and then copying it all back into a ZIP (JAR) to just copying everything into a ZIP (JAR) directly -- which is why 0.1.7 is so much slower than 0.2.x. It now uses a zip filesystem directly. I wasn't aware that changed the memory usage but it's certainly possible. All our uberjars are 30-40MB each and we haven't seen any memory issues using the default heap settings running clj... #2019-07-18 22:45:52 kenny The uberjar for the service with this issue is 60mb. #2019-07-18 22:47:41 kenny It definitely has to do with java 11. I reverted our build and prod images to java 8 and everything builds as expected - no mem issue. #2019-07-18 22:47:43 seancorfield The change also served as a solution for Windows-related issues with reserved filenames (so you couldn't run depstar 0.1.7 at all on Windows when you depended on libraries that contained certain filenames!). #2019-07-18 22:48:22 seancorfield Good to know. We're on Adopt OpenJDK8 everywhere because some stuff we rely on won't run on JDK11. #2019-07-18 22:51:40 kenny We've been using CircleCI's pre-built images for building uberjars. Specifically circleci/clojure:openjdk-11-tools-deps-1.10.0.442 is the one that produces the mem issue. Not sure if they use adopt openjdk or not. #2019-07-19 08:20:01 just.sultanov @U083D6HK9 we use the latest versions and the project works on openjdk 8,9,11,12:
http-kit {:mvn/version "2.4.0-alpha4"} ;; fixes java 11
javax.xml.bind/jaxb-api {: mvn / version "2.3.1"} ;; support java 9+
org.clojure/core.rrb-vector {: mvn / version "0.0.14"} ;; support java 11+
see more info about http-kit: https://github.com/http-kit/http-kit/releases/tag/v2.4.0-alpha3 #2019-07-19 15:24:24 kenny @U1EQNSHL4 Not sure I understand what you mean. This issue was about building an uberjar with depstar, not an issue with http-kit. #2019-07-20 01:05:11 steveb8n Q: has anyone seen and fixed this warning “SLF4J: Class path contains multiple SLF4J bindings.” #2019-07-20 01:05:42 steveb8n for the life of me I can’t figure out how to add the deps exclusion correctly in deps.edn #2019-07-20 02:50:03 alexmiller In the coordinate, add :exclusions [group/artifact] #2019-07-20 06:31:32 steveb8n thanks but I need to know which dep to exclude. I should have been clearer in my question. #2019-07-20 06:31:53 steveb8n here’s the warning I see
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/steve/.m2/repository/com/fzakaria/slf4j-timbre/0.3.13/slf4j-timbre-0.3.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/steve/.m2/repository/org/slf4j/slf4j-nop/1.7.7/slf4j-nop-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [com.github.fzakaria.slf4j.timbre.TimbreLoggerFactory]
#2019-07-20 06:34:01 steveb8n I’m want to use timbre so this suggests I should exclude the slf4j-nop dep. when I examine my deps tree, that is a transitive dep from Clojure (via tools.deps). I can’t figure out how to exclude that completely, or if there’s a better way to do this? #2019-07-20 08:06:07 dominicm @U0510KXTU something is wrong if your clojure depends on slf4j-nop #2019-07-20 08:06:15 dominicm do you mean something other than org.clojure/clojure? #2019-07-20 08:07:23 steveb8n let me double-check…. #2019-07-20 08:08:48 steveb8n quite right, it’s coming from datomic-free #2019-07-20 08:09:13 dominicm datomic-free probably shouldn't depend on slf4j-nop, but yeah. #2019-07-20 08:11:53 steveb8n but there’s two more: from tools.gitlibs and tools.deps #2019-07-20 08:12:08 steveb8n I can’t see how they are getting in there #2019-07-20 08:13:35 steveb8n I take it back. the extra one comes as part of tools.deps #2019-07-20 08:14:10 steveb8n is there a way to explicitely exclude that? maybe explicitely add tools.deps as a dep? #2019-07-20 08:15:30 dominicm I think you just need to change your t.d.a dependency to have an exclusion of slf4j-nop? #2019-07-20 08:16:05 steveb8n I didn’t have a t.d.a dep in my deps #2019-07-20 08:16:26 dominicm Where's it coming from? #2019-07-20 08:16:34 steveb8n I added one with the exlusion (same as datomic-free) but it’s still there for some reason. digging…. #2019-07-20 08:17:42 dominicm is this a cursive issue, i.e. does it not appear in clojure -Stree? #2019-07-20 08:18:10 steveb8n I think it might be. good idea, I’ll check from the CLI #2019-07-20 08:19:17 steveb8n quite right, not a problem from the CLI #2019-07-20 08:19:54 dominicm I guess cursive is adding t.d.a to your deps for some reason? #2019-07-20 08:20:44 steveb8n you legend! despite the presence in Cursive, the warning has gone at load time. that fixed it #2019-07-20 08:21:00 steveb8n once again, you’ve helped me. thanks mate #2019-07-20 08:21:20 steveb8n if you are at Clojutre, I’ll buy you a beer #2019-07-20 08:28:06 dominicm Unfortunately not, I'm at Heart of Clojure though on the chance you are, I know you struggle to get up for the european ones. I'm sure my fellow JUXTers will enjoy the beer though 🙂 #2019-07-20 08:39:26 steveb8n I’ll pay it forward. I just listened to the Juxt-cast today and heard about XT20. might have to switch to that next year #2019-07-23 18:45:49 ticean Is there a way to control HTTP timeouts or retries with Maven dependencies when using tools-deps? I must have a shakey connection because I’m getting HTTP errors when performing clojure -Stree.
With enough retries I can get them all, but this isn’t working very well my build process. Any suggestions? #2019-07-23 18:47:30 alexmiller no support for that, sorry #2019-07-23 18:48:14 alexmiller I'm not even sure what Maven offers in the apis for that, but nothing is surfaced through tools.deps #2019-07-23 18:53:03 ticean Ok, thanks for the response! Google tells me that there’s some Maven HTTP configurations through XML, but I have very little knowledge in this area. Most of my Maven interactions have been through Clojure tooling. 🙂 #2019-07-23 18:53:25 ticean I’ll try in the build server. Hopefully the probs are local to my network. #2019-07-23 18:55:09 alexmiller XML in pom.xml or in settings.xml? #2019-07-23 18:59:11 alexmiller yes, that's settings.xml, but this is for wagons, which is the "old stuff". tools.deps uses the newer Maven resolver with transports instead #2019-07-23 18:59:43 alexmiller I doubt anything we're doing in tools.deps is looking at it right now (although honestly I'm not certain) #2019-07-23 19:01:05 ticean Ah, I see. Thanks. I figured I was not getting something because I don’t know the ecosystem very well. #2019-07-23 19:02:19 alexmiller we're using org.eclipse.aether.transport.http.HttpTransporter, which does seem to have some sort of configuration facility for http parameters. how to actually get that configured would take some spelunking #2019-07-23 19:04:21 alexmiller I kinda see how it could be done #2019-07-23 19:08:46 ticean 👍 Thanks. That’s understandable. Dependency pulls have worked well for me too, until my summer relocation to a cabin in the woods. Has been great for hammocking, now just trying to get those dependencies. 🙂 #2019-07-23 19:19:12 alexmiller you just need to mirror maven central and clojars before heading to the woods #2019-07-23 20:30:29 avi I’m using Cambada in my project to do AOT and create an überjar. It’s been working well for me. #2019-07-23 23:04:18 hiredman the clj script I have (version 1.9.0.375 I guess? from the -Sdescribe, not sure how to track that to a tools.dep version) doesn't seem to handle '-i' in what I thought was the correct way. my understanding is -i means "load this file and give me a repl", and what happens is it loads the file and then exits #2019-07-23 23:07:31 seancorfield You need -r to get a REPL if you specify any of the main opts I think. #2019-07-23 23:07:52 seancorfield (and it's still true on the latest version of the clj script I'm fairly sure) #2019-07-23 23:08:58 seancorfield Confirmed: (! 693)-> clj -Sdescribe
{:version "1.10.1.466"
...
Tue Jul 23 16:08:19
(sean)-(jobs:0)-(~/clojure)
(! 694)-> clj -i script.clj
Hello from script
Tue Jul 23 16:08:29
(sean)-(jobs:0)-(~/clojure)
(! 695)-> clj -i script.clj -r
Clojure 1.10.1
Hello from script
user=> #2019-07-23 23:09:27 hiredman ah, I wonder if that has always been the case and I just forgot about it because of how little I use -i #2019-07-23 23:10:39 hiredman looks like it #2019-07-23 23:22:47 alexmiller That has been the same forever #2019-07-24 19:21:41 Travis just curious but is it possible to pull dependencies from a Google Cloud Storage based maven repo ? #2019-07-24 19:27:57 alexmiller if it acts as a Maven repo on http, then sure #2019-07-24 19:28:36 alexmiller if not, then no #2019-07-24 19:28:44 alexmiller how do you configure it in Maven? #2019-07-24 19:29:41 alexmiller if there's a custom wagon, then that will not work #2019-07-24 19:30:05 alexmiller but it could be made to do so #2019-07-24 19:34:12 Travis need to research more. Currently I have a jar that is being built/published by gradle into GCS using there maven-publish plugin. Basically to avoid managing some private artifact repository and just use GCS #2019-07-24 19:45:13 alexmiller sure, this is commonly done with s3 (and tools.deps includes support for that) #2019-07-24 21:50:51 seancorfield I'm modifying depstar to insert a MANIFEST.MF file and pom.xml / pom.properties. My minimalist thinking is that it should do this automatically if it finds a pom.xml file in the directory in which it is run -- as part of the project you are building a JAR/uberjar for. Is this a good idea and if not, why not? #2019-07-24 21:52:46 seancorfield I'm thinking you'd do clj -Spom && clj -A:jar my-project.jar (assuming the :jar alias brings in depstar and uses -m hf.depstar.jar). #2019-07-24 21:53:57 seancorfield Are there situations where automatically picking up the local pom.xml file is a bad idea? Are there situations where you would want to use a different file as the POM or you might have the POM in a different location? #2019-07-24 21:55:49 dorab Not directly answering your question, but any changes to depstar that allow me to java -jar foo.jar rather than java -cp foo.jar clojure.main -m foo.core would be useful. #2019-07-24 22:00:57 seancorfield @dorab that would require depstar to deal with AOT -- not happening. #2019-07-24 22:01:34 seancorfield What this change will allow for is java -jar foo.jar -m your.entry.point instead of java -cp foo.jar clojure.main -m your.entry.point #2019-07-24 22:03:03 seancorfield (assuming you had pom.xml locally -- so depstar would generate a manifest and because you're generating an uberjar, instead of a library JAR, you'd get Main-Class: clojure.main added) #2019-07-24 22:06:54 seancorfield I guess my real question is whether it would be useful/important to add a flag to suppress consumption of pom.xml? #2019-07-24 22:15:47 seancorfield (easy enough to do, so I might as well) #2019-07-24 22:31:14 alexmiller Java doesn’t care about the pom stuff so that seems harmless. clj will use it to find dependencies though if you use it as a local file dep #2019-07-24 22:31:46 alexmiller That’s prob good if not an uberjar and bad if it is an uberjar #2019-07-24 22:33:19 alexmiller (As you’d get deps from two places on the classpath) #2019-07-24 22:40:04 seancorfield Yeah, that's why this has cropped up -- http://cljdoc.org assumes the JAR contains it. #2019-07-24 22:40:46 seancorfield And with that ... seancorfield/depstar "0.3.0" is available! #2019-07-24 23:44:17 dorab @seancorfield java -jar foo.jar -m your.entry.point is good enough for me. Thanks for the release. Will try it out. #2019-07-25 00:41:37 ghadi @seancorfield is the main class automatically inserted into the jar now? (Haven't tried it) #2019-07-25 00:42:43 seancorfield If you build an uberjar, it adds Main-Class: clojure.main #2019-07-25 00:43:07 seancorfield Since it does no AOT, you still have to go through clojure.main. #2019-07-28 16:20:49 borkdude local/root is only used for development on a local machine? e.g. you cannot specify a local/root dep in a git dep?
example:
project/examples/deps.edn <- used as git dep, refers to library on level up as :local/root ".."
project/deps.edn <- library #2019-07-28 17:43:37 alexmiller With the most recent set of dir related fixes, I think that should work #2019-07-28 17:43:44 alexmiller Did you try it? #2019-07-28 18:15:40 borkdude I did try it, let me see if I have the latest #2019-07-28 18:25:20 borkdude upgraded and I can confirm this works. this is awesome, thanks #2019-07-28 18:25:29 alexmiller Cool #2019-07-28 18:43:47 dominicm there was another set of changes...? #2019-07-28 18:45:39 borkdude it seems so 🙂 #2019-07-28 18:46:17 borkdude the version I'm using right now is 1.10.1.466 #2019-07-28 18:52:33 dominicm Nothing in the changelog indicates tdeps-132 is resolved. I think this is still an issue. #2019-07-28 18:53:25 dominicm Essentially if you have 2 git dependencies in that repo, that both have a :local/root on C, then you'll hit an error: https://clojure.atlassian.net/browse/TDEPS-123
This is important for people who want to use Edge without cloning it. #2019-07-28 18:53:49 borkdude :local/root on C? #2019-07-28 18:54:37 dominicm given deps, A, B, C:
deps.edn
- A {:git/url ...}
- C {:local/root "../C"}
- B {:git/url ...}
- C {:local/root "../C"}
#2019-07-28 18:55:54 borkdude ah #2019-07-28 21:31:53 carkh hum cannot access this link if logged in service desk #2019-07-28 21:32:08 carkh works when logged out though #2019-07-28 21:32:20 carkh (not a deps issue) #2019-07-29 10:59:50 borkdude @dominicm good to know that doesn't work yet - if it did, that would be a reason for me to jump from boot to tools.deps in our local codebase, since we have locally shared deps now that we must deploy somewhere, which feels a bit icky #2019-07-29 14:13:07 alexmiller as TDEPS-132 indicates, I consider this out of scope for now, no plans to work on that #2019-07-29 17:57:28 borkdude is there something in tools.deps which combines R and C? so I want all test deps but also the test dir to be on the classpath #2019-07-29 17:57:59 borkdude so clojure -Rtest -Ctest -Spath #2019-07-29 17:58:48 alexmiller -A #2019-07-29 18:05:44 borkdude but won't that also execute a main? #2019-07-29 18:14:32 alexmiller yes, if there is one in the test alias. you didn't mention that #2019-07-29 18:15:18 alexmiller if that's the case, then I'd say no, use -R and -C #2019-07-29 18:16:00 borkdude well, I only know after looking, so it's not something I would use by default that way #2019-07-29 18:16:44 borkdude and people might add one later, so if I do that in a script, the script might break #2019-07-29 18:16:54 borkdude so -R and -C is fine #2019-07-29 23:05:17 kenny Every time I run clj in some of my projects, I get all of this outputted:
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-netty-shaded/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Is this necessary? #2019-07-29 23:12:46 seancorfield @kenny That indicates you are either depending on SNAPSHOT releases or RELEASE/LATEST rather than specific non-snapshot versions. #2019-07-29 23:13:24 kenny That would surprise me. Would that show up in -Stree? #2019-07-29 23:13:39 seancorfield Since snapshots and both RELEASE/LATEST can refer to different versions over time, the dependency checker must fetch metadata to check which version is current. #2019-07-29 23:13:51 seancorfield Yeah, I would expect it to show up in -Stree #2019-07-29 23:14:18 kenny Nothing shows up using any of those identifiers. #2019-07-29 23:15:42 seancorfield Well, something is trying to pull in io.grpc/grpc-core and/or io.grpc/grpc-api... #2019-07-29 23:16:05 kenny Correct but -Stree shows the version locked. #2019-07-29 23:16:34 seancorfield -Stree shows what version it is resolved to -- not necessarily how it was declared. #2019-07-29 23:17:11 seancorfield So using the output of -Stree you can figure out what path through the dependencies was taken -- and then look in your deps.edn to see how you're pulling those top-level things in... #2019-07-29 23:17:35 kenny Well it's from this dep: com.google.cloud/google-cloud-monitoring {:mvn/version "1.78.0"} #2019-07-29 23:19:12 kenny I think it'd be weird for that lib to use one of those identifiers. #2019-07-29 23:20:16 kenny Ah, was just ctrl+f'ing. That's a weird way to do it. #2019-07-29 23:20:38 kenny So because they didn't specify a version, tools-deps tries to find the latest one each time? #2019-07-29 23:20:43 seancorfield Yup. #2019-07-29 23:21:12 kenny Bleh. Why would they do that? #2019-07-29 23:21:33 seancorfield Because they're Google? You can probably fix that by putting those dependencies explicitly in your project with appropriate versions... #2019-07-29 23:23:07 seancorfield So maybe tools.deps doesn't follow that path? @alexmiller might be able to speak to that. #2019-07-29 23:23:30 seancorfield But at least that gives the specific versions you'd be expected to use (with GCM 1.78.0) #2019-07-29 23:24:36 kenny Is a parent supposed to be included in the dep tree or only used to resolve artifacts in the child? #2019-07-29 23:24:55 seancorfield (look at the <properties> section of that parent pom I linked to) #2019-07-29 23:25:06 seancorfield That question is beyond my Maven knowledge, sorry. #2019-07-29 23:26:15 kenny So weird. Everything works as expected in the REPL. #2019-07-29 23:26:41 seancorfield in the clj repl? #2019-07-29 23:26:54 kenny Yep #2019-07-29 23:27:03 seancorfield Right, because it resolves all the versions. #2019-07-29 23:28:05 seancorfield I'm just suggesting you can suppress those metadata fetches by explicitly declaring dependencies in deps.edn on the relevant versions of io.grpc/grpc-core, io.grpc/grpc-api, etc. #2019-07-29 23:28:51 seancorfield I have a bunch of dev-only tooling where I use RELEASE as the version and every day clj will show messages about fetching metadata -- to check the most recent version. #2019-07-29 23:29:32 seancorfield In your -Stree output, what version of those io.grpc libs does it show that it resolves to? #2019-07-29 23:29:52 kenny 1.21.0 #2019-07-29 23:33:24 kenny Adding io.grpc/grpc-core {:mvn/version "1.21.0"} fixes this. #2019-07-29 23:33:47 seancorfield Yup. That's what I'd expect. #2019-07-31 02:45:07 minikomi well, it took a little while, but i'm super sold on deps.edn
i had to write my own watch/reload logic for parts i was working on.. but it was actually fairly painless using ns-tracker
love the ease of switching between maven/git/local deps when developing libraries .. really encourages breaking useful parts out sooner rather than later #2019-07-31 02:45:50 minikomi i tend to use mount a lot more too to get my garden/cljs/web server/nrepl scaffolding up and running & reloadable #2019-07-31 02:47:17 carkh yes the local deps are so helpful for exercising an ongoing library project #2019-07-31 02:48:19 minikomi docs for some things like using figwheel as part of a project rather than straight from the command line are a bit thin, but i fell much more in tune with my stack now that i've ventured deeper down and built out the parts i need haha #2019-08-03 13:19:03 sekao am i right that there is no way to combine aliases in deps.edn? so instead of writing clj -A:one:two i could just write clj -A:two and it would pull in :one automatically. i know you can't pass aliases to :main-opts so i'm guessing i just need to write them both explicitly #2019-08-03 14:22:46 alexmiller correct #2019-08-03 20:39:53 dharrigan Any pointers to approaches/libraries that are similar to the intent of pomegranate/alembic to allow for reloading of deps.edn within a running repl (to pull in a new dep) or to dynamically add in a dep on the repl (to kickstart a download and adding to the classpath)? #2019-08-03 20:49:21 seancorfield @dharrigan tools.deps has a branch with add-lib that dynamically adds new libraries to the running system... #2019-08-03 20:49:38 dharrigan sweet, ta @seancorfield! #2019-08-03 20:50:09 dharrigan I guess my next question is (when) will it be merged in and released? #2019-08-03 20:50:52 dharrigan kk #2019-08-03 20:51:12 dharrigan pretty awesome 🙂 #2019-08-03 20:51:17 seancorfield That also has an example of how to add master from GitHub 🙂 #2019-08-03 20:51:53 dharrigan thanks 🙂 #2019-08-05 19:52:33 seancorfield seancorfield/depstar {:mvn/version "0.3.1"} was just released -- includes a new -m / --main option so that if you have a pom.xml and do your own AOT/compile step to create a "main" entry point, you can now override the Main-Class in the generated manifest (overriding the default of clojure.main). #2019-08-05 19:56:07 ghadi depstar @seancorfield? #2019-08-05 19:56:33 ghadi the coord might be missing a piece #2019-08-05 20:11:03 seancorfield Updated! Thanks! #2019-08-06 12:24:55 dharrigan So #2019-08-06 12:25:28 dharrigan Pulling in tools.deps.alpha via a git/url hash, to access the branch that allows me to load in a jar from the repl #2019-08-06 12:25:38 dharrigan I'm also using slf4j #2019-08-06 12:25:47 dharrigan My logs stopped working (i.e., output) #2019-08-06 12:26:08 dharrigan SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/david/.m2/repository/org/slf4j/slf4j-nop/1.6.2/slf4j-nop-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/david/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
#2019-08-06 12:26:20 dharrigan according to the doc, the choice is totally random #2019-08-06 12:26:33 dharrigan why does deps include the nop static binding? #2019-08-06 12:26:56 dharrigan I can't exclude it, like I can if I'm doing "mvn/version" #2019-08-06 12:27:02 dharrigan (using :exclusions) #2019-08-06 12:27:42 dharrigan org.clojure/tools.deps.alpha d0b33e0
...
...
chopped for brevity
...
org.slf4j/slf4j-nop 1.6.2
#2019-08-06 12:28:10 dharrigan From the slf4j doc #2019-08-06 12:28:12 dharrigan The warning emitted by SLF4J is just that, a warning. Even when multiple bindings are present, SLF4J will pick one logging framework/implementation and bind with it. The way SLF4J picks a binding is determined by the JVM and for all practical purposes should be considered random. As of version 1.6.6, SLF4J will name the framework/implementation class it is actually bound to.
#2019-08-06 12:29:36 alexmiller I think we have a ticket for this, probably need to stop including that #2019-08-06 12:30:00 dharrigan Yes please! Having a library link in the static binding ain't right 🙂 #2019-08-06 12:30:20 alexmiller Why did the exclusion not work? #2019-08-06 12:30:29 dharrigan not supported, according to the docs: #2019-08-06 12:31:01 dharrigan you can for maven coordinates, but not for git coordinates #2019-08-06 12:31:14 alexmiller Did you try it? #2019-08-06 12:31:17 dharrigan Yup #2019-08-06 12:31:32 dharrigan org.clojure/tools.deps.alpha {:git/url ""
:sha "d0b33e0d346736aa985c150145b332f97b92135e"
:exclusions ["org.slf4j/jcl-over-slf4j"
"org.slf4j/slf4j-nop"] }}}
#2019-08-06 12:31:50 alexmiller Should be a symbol, not a string #2019-08-06 12:32:12 alexmiller I think it will work #2019-08-06 12:32:32 dharrigan ah good spot #2019-08-06 12:32:35 dharrigan retrying #2019-08-06 12:33:33 dharrigan that worked, after rm -rf'ing my .cpcache dir #2019-08-06 12:34:19 dharrigan ta very much, but still, I think the nop should be excluded totally, may catch others out 😉 #2019-08-06 12:34:27 alexmiller Yeah #2019-08-06 12:34:46 dharrigan parrot #2019-08-06 13:15:51 rickmoynihan #2019-08-06 13:17:24 rickmoynihan Her ssh key appears to be the same version as mine which works…
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,xxxx
#2019-08-06 13:18:03 rickmoynihan she’s also running ssh-agent and has added the key #2019-08-06 13:44:41 alexmiller so ssh-add -l lists her identity? #2019-08-06 13:45:33 alexmiller and I assume she does actually have access to the git repo (can access via git) #2019-08-06 13:48:16 alexmiller I've found on mac that I had a setting in my ~/.ssh/config under "Host *" that had a setting for IdentityFile or something like that which I had to delete #2019-08-06 13:48:18 rickmoynihan yes to both of those #2019-08-06 13:48:43 rickmoynihan 👀 #2019-08-06 13:49:04 alexmiller my ssh config looks like: #2019-08-06 13:49:08 alexmiller Host *
AddKeysToAgent yes
UseKeychain yes
#2019-08-06 13:49:35 rickmoynihan thanks @alexmiller dropping IdentityFile works #2019-08-06 13:49:50 alexmiller ssh ninja #2019-08-06 13:50:55 rickmoynihan :punch: #2019-08-06 13:51:48 alexmiller I added that to that jira issue in a comment #2019-08-06 13:52:25 rickmoynihan 👍 my future self with thank you again for that #2019-08-06 13:56:06 alexmiller ditto, I had to look it up (again) #2019-08-06 14:21:58 rgm @alexmiller 🎉 … private deps seem to be working for me now #2019-08-06 14:25:07 rgm (to be clear, this is just a general man-are-there-ever-a-lot-of-moving-parts-to-auth gripe, not a tools.deps gripe 🙂) #2019-08-06 18:24:31 zane I'm curious how y'all are switching between different Java versions when using clj and clojure. #2019-08-06 18:25:09 alexmiller hint: the answer has nothing to do with clj and clojure :) #2019-08-06 18:25:23 alexmiller they just use whatever java is on the path #2019-08-06 18:26:23 alexmiller so to rephrase, how are y'all switching between Java versions on your OS of choice? #2019-08-06 18:28:03 zane > they just use whatever java is on the path
Got it. So that means that clj and clojure will ignore the version set by, for example, jenv? #2019-08-06 18:30:09 zane I suppose since jenv works by providing a java shim it ought to still work provided that the jenv shim is the first java on the path. 🤔 #2019-08-06 18:31:51 alexmiller I don't know what jenv does but that seems right #2019-08-06 18:40:34 zane Thank you! 🙂 #2019-08-06 19:17:02 seancorfield @zane I set JAVA_HOME to point to the JDK installation that I want to use... (! 586)-> JAVA_HOME=$OPENJDK11_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"11.0.1"
user=> ^D
Tue Aug 06 12:16:46
(sean)-(jobs:0)-(~/clojure)
(! 587)-> JAVA_HOME=$OPENJDK8_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"1.8.0_192"
user=>
#2019-08-06 19:17:02 seancorfield @zane I set JAVA_HOME to point to the JDK installation that I want to use... (! 586)-> JAVA_HOME=$OPENJDK11_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"11.0.1"
user=> ^D
Tue Aug 06 12:16:46
(sean)-(jobs:0)-(~/clojure)
(! 587)-> JAVA_HOME=$OPENJDK8_HOME clj
Clojure 1.10.1
user=> (System/getProperty "java.version")
"1.8.0_192"
user=>
#2019-08-06 21:26:26 zane Hey Sean, thanks for sharing your process! This only changes the JAVA_HOME for the command that follows, right? (Or for the shell that does the export if export is included.) #2019-08-06 21:27:54 seancorfield Just for that command, yes. Which is why it's nice for testing. #2019-08-06 21:29:58 zane Makes a ton of sense for testing. I've done that myself. #2019-08-06 21:30:01 zane The particular use case I'm solving for is when I have a long-running Emacs process that occasionally launches clojure processes via cider-jack-in. #2019-08-06 21:30:30 zane In that case setting JAVA_HOME in the shell won't result in it getting picked up by the already-running Emacs process. #2019-08-06 22:47:06 seancorfield But you could get Emacs to set the env var for the invocation of clojure by changing the command line it uses? #2019-08-06 22:59:20 zane If I'm understanding you correctly, probably? Would require delving deeper into Cider than I want to do right now. 🙂 #2019-08-07 14:04:41 Ahmed Hassan @zane you can connect to running clojure process by using cider-connect-clj #2019-08-07 14:05:11 Ahmed Hassan Similarly for ClojureScript you can use cider-connect-cljs #2019-08-07 14:15:33 zane Yes, I'm aware. I prefer to let Cider launch Clojure because choosing compatible versions of all the middleware can be difficult. But thank you! 🙂 #2019-08-07 16:04:04 seancorfield "because choosing compatible versions of all the middleware can be difficult" -- a big part of why I stopped using Emacs/CIDER/nREPL #2019-08-07 16:11:08 Ahmed Hassan @U04V70XH6 What do you use now? #2019-08-07 16:11:47 seancorfield I switched from Emacs to Atom/ProtoREPL and then to Atom/Chlorine. I can connect Atom/Chlorine to any Clojure process since it only requires a Socket REPL. #2019-08-07 16:12:42 seancorfield (so I can, and do, connect Atom/Chlorine to QA and production processes that were configured to start a Socket REPL using JVM options -- no dependencies needed in the project, no logic to start a custom REPL) #2019-08-07 16:13:37 Ahmed Hassan How is experience of Atom/Chlorine as compared with Cider/Emacs? #2019-08-07 17:19:14 zane Yeah. I can understand why people make that switch, though for me it wouldn't be worth it. #2019-08-06 20:01:28 dominicm I think the way voidlinux does it is neat. JAVA_HOME points at a symlink, and xbps-alternatives changes the symlink. #2019-08-06 21:08:24 dharrigan I use arch, so I do sudo archlinux-java set java-12-openjdk #2019-08-06 21:08:40 dharrigan or sudo archlinux-java set java-8-openjdk #2019-08-06 21:08:45 dharrigan easy peasy #2019-08-07 01:12:13 cfleming If I want to use a fork of Clojure with deps, can I exclude the clojure version pulled in from the system deps.edn somehow? #2019-08-07 04:13:31 seancorfield @cfleming If your local deps.edn file has that (local?) Clojure in :deps directly, it should override the system deps anyway? #2019-08-07 05:19:09 alexmiller correct ^^ #2019-08-07 05:19:45 alexmiller assuming you have built and installed that fork locally in Maven so that it's artifact id is still org.clojure/clojure #2019-08-07 05:20:12 alexmiller if you want to call it something else (or use as a git or local dep), then no, that is not currently supported #2019-08-07 05:22:55 alexmiller that's not really intentional just kind of a consequence of various assumptions #2019-08-07 05:23:21 cfleming Yeah, I changed the artifact ID to reduce confusion. #2019-08-07 05:23:29 cfleming Hmm #2019-08-07 05:23:34 alexmiller top-level exclusions would possibly help with this but not there yet #2019-08-07 05:23:45 cfleming Ok, I think it would be kind of a pain to use due to… right, the lack of that. #2019-08-07 05:24:00 cfleming I’ll stick with Maven for now then, thanks! #2019-08-07 05:25:26 cfleming Actually, if I had the forked jar in a lib dir, and used :deps {org.clojure/clojure {:local/root "lib/clojure-1.10.1.jar"}}, that should work, right? #2019-08-07 05:25:52 cfleming i.e. deps will assume that the artifact ID is whatever I tell it. #2019-08-07 05:26:14 cfleming And that will override the official one. #2019-08-07 05:35:42 alexmiller there are some issues with overriding a maven dep with a git / local dep #2019-08-07 05:35:54 alexmiller so I don't think it will work #2019-08-07 05:36:09 cfleming Ok, thanks. I’ll ditch that idea for now, then. #2019-08-07 05:36:27 alexmiller that one kind of bugs me. I haven't had a chance to look at it yet #2019-08-07 05:36:44 alexmiller part of it is the version comparison across dep types #2019-08-07 05:37:03 alexmiller but on override that shouldn't matter #2019-08-07 20:56:13 hkupty Hi folks, has anyone successfully used tools.deps w/ git repos on private repositories? #2019-08-07 20:59:48 alexmiller Yes #2019-08-07 21:12:24 hkupty As the maintainer of both tools.deps and tools.gitlibs, I expected a more user-friendly answer from you. #2019-08-07 21:13:11 alexmiller Sorry, was just answering your question. Was hard to provide any additional detail without any problem description #2019-08-07 21:02:27 hkupty I'm having issues w/ unknown host key if I use {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "6f08061b2f414141"}, :content ("[email protected]")} or I need to provide credentials if I use the https form, which I obviously won't do, since I'm not comiting credentials on the repo #2019-08-07 21:02:46 hkupty Is there any docs on how to get over that? #2019-08-07 21:13:42 alexmiller can you share the error message or more info about your setup and what you're doing? #2019-08-07 21:13:52 alexmiller what os are you on? #2019-08-07 21:15:28 alexmiller https auth is not currently supported so you'll need to use ssh #2019-08-07 21:16:11 hkupty I'm running from Fedora 30, very basic setup.
The error code I get is:
Execution error (JSchException) at com.jcraft.jsch.Session/checkHost (Session.java:805).\nUnknownHostKey: . RSA key fingerprint is .....
It came from a direct invocation of tools.deps respolve-deps:
(deps/resolve-deps {:deps {'project {:git/url " #2019-08-07 21:17:48 hkupty I'm running the repl from a session where I've added the keys to ssh-agent, so I assume this step is done. #2019-08-07 21:18:17 carkh did you add the host to your .ssh/known_hosts ? #2019-08-07 21:18:47 hkupty Hum... I think I know what could be the issue. #2019-08-07 21:19:02 hkupty I'm using ED25519 instead of RSA for gitlab. #2019-08-07 21:19:59 ghadi ssh-keyscan -t rsa,ed25519 >> ~/.ssh/known_hosts #2019-08-07 21:20:20 ghadi that will seed UnknownHostKeys ^ into your config #2019-08-07 21:21:04 ghadi ED25519 on the user identity file is fine (when using an agent) @ingvij
the error you're pasting is an rsa host key, a different kind of key #2019-08-07 21:22:41 hkupty I see. Fine then, I'll try both approaches and I'll post the results back here. #2019-08-07 21:22:45 ghadi both? #2019-08-07 21:23:21 hkupty ssh-keyscan and the link @alexmiller posted #2019-08-07 21:23:24 ghadi the article @alexmiller posted seems to use the same exact thing I'm describing #2019-08-07 21:23:52 hkupty ahh, sorry. I hadn't read the article yet #2019-08-07 21:23:53 ghadi since clojure deps.edn is not terminal interactive, the .ssh/known_hosts file needs the host key #2019-08-07 21:24:14 hkupty Right, that makes sense. #2019-08-07 21:25:02 ghadi (but this approach is a bit insecure because no one is validating the fingerprints...) #2019-08-07 21:25:31 ghadi (then again most people type Yes anyways when ssh prompts you..) #2019-08-07 21:27:26 hkupty Should work fine as a workaround I suppose. #2019-08-07 21:29:02 hkupty That worked for checking the dependency, but I'm getting a MetadataNotFoundException now. I'll double-check everything, specially the dependency itself. Thanks for the help @ghadi and @alexmiller. #2019-08-07 21:31:37 alexmiller if you can share more info, might have some idea what that is, but I don't even know what layer of the stack it's from #2019-08-07 21:32:23 alexmiller just in general, providing information at the beginning lets people help you more efficiently, and we're all doing this for free, in spare time #2019-08-07 21:32:48 hkupty Execution error (MetadataNotFoundException) at org.eclipse.aether.internal.impl.DefaultMetadataResolver/resolve (DefaultMetadataResolver.java:233).\nCould not find metadata org.clojure:clojure/maven-metadata.xml in local (/home/hkupty/.m2/repository)
#2019-08-07 21:36:16 hkupty The stacktrace from tools.deps:
[[org.eclipse.aether.internal.impl.DefaultMetadataResolver
resolve
"DefaultMetadataResolver.java"
233]
[org.eclipse.aether.internal.impl.DefaultMetadataResolver
resolveMetadata
"DefaultMetadataResolver.java"
191]
[org.apache.maven.repository.internal.DefaultVersionResolver
resolveVersion
"DefaultVersionResolver.java"
233]
[org.eclipse.aether.internal.impl.DefaultRepositorySystem
resolveVersion
"DefaultRepositorySystem.java"
238]
[clojure.tools.deps.alpha.extensions.maven$eval362$fn__365
invoke
"maven.clj"
40]
[clojure.lang.MultiFn invoke "MultiFn.java" 239]
[clojure.tools.deps.alpha$canonicalize_deps$fn__761
invoke
"alpha.clj"
68]
[clojure.core.protocols$fn__8159 invokeStatic "protocols.clj" 168]
[clojure.core.protocols$fn__8159 invoke "protocols.clj" 124]
[clojure.core.protocols$fn__8114$G__8109__8123
invoke
"protocols.clj"
19]
[clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 31]
[clojure.core.protocols$fn__8144 invokeStatic "protocols.clj" 75]
[clojure.core.protocols$fn__8144 invoke "protocols.clj" 75]
[clojure.core.protocols$fn__8088$G__8083__8101
invoke
"protocols.clj"
13]
[clojure.core$reduce invokeStatic "core.clj" 6828]
[clojure.core$reduce invoke "core.clj" 6810]
[clojure.tools.deps.alpha$canonicalize_deps
invokeStatic
"alpha.clj"
67]
[clojure.tools.deps.alpha$canonicalize_deps invoke "alpha.clj" 65]
[clojure.tools.deps.alpha$expand_deps$fn__787
invoke
"alpha.clj"
184]
[clojure.tools.deps.alpha$expand_deps invokeStatic "alpha.clj" 183]
[clojure.tools.deps.alpha$expand_deps invoke "alpha.clj" 164]
[clojure.tools.deps.alpha$resolve_deps invokeStatic "alpha.clj" 231]
[clojure.tools.deps.alpha$resolve_deps invoke "alpha.clj" 213]
#2019-08-07 21:38:49 alexmiller org.clojure:clojure/maven-metadata.xml looks weird #2019-08-07 21:38:56 alexmiller maybe bad format in your deps.edn? #2019-08-07 21:39:08 alexmiller org.clojure:clojure when should be org.clojure/clojure maybe? #2019-08-07 21:43:45 ghadi boo #2019-08-07 21:44:30 ghadi you should choose an explicit version for clojure #2019-08-07 21:45:42 ghadi @alexmiller maybe -Sresolve can rewrite symbolic maven things, too #2019-08-07 21:46:02 hkupty I know. this is still on very initial stages, just testing really.. #2019-08-07 21:46:24 hkupty That's why I didn't care too much about it. Just pinned the version. #2019-08-07 21:47:41 alexmiller re RELEASE, agreed, bleh #2019-08-07 21:48:28 alexmiller RELEASE is not officially supported in deps.edn #2019-08-07 21:48:58 alexmiller I've let it continue to work as it's sometimes a good hack, but shouldn't be enshrining it #2019-08-07 21:50:08 alexmiller -Sresolve is going to go away (move into an external tool) #2019-08-07 21:53:57 hkupty I moved past that error, now yet another one bothering me. A dependency of a dependency is not being resolved, whereas if I run clj on the library I'm trying to import, it works fine:
Syntax error (ExceptionInfo) compiling at (cli/ops.clj:5:1).\nUnable to resolve io.grpc/grpc-core version: [1.17.1]
When I run the repl directly it works, if I call this it fails:
(deps/resolve-deps {:deps {'io.etcd/jetcd-core {:mvn/version "0.3.0"}}}
{})
#2019-08-07 21:55:13 hkupty I don't want to keep you involved in the debugging unless you want to, I just keep posting since I think those might be relevant. I'd understand if you'd rather want me to open an issue. #2019-08-07 21:56:03 hkupty which, btw, I'll do for clj-new so new projects are not created w/ RELEASE on clojure version. #2019-08-07 21:57:12 hiredman that is a syntax error, not related to deps at all #2019-08-07 21:57:54 hiredman I would dig in to cli/ops.clj on the given line number, likely something isn't being quoted correctly #2019-08-07 22:00:51 hkupty that particular invocation is this one:
(deps/resolve-deps {:deps {'io.etcd/jetcd-core {:mvn/version "0.3.0"}}}
{})
I believe this might be due to some parsing from pom.xml instead of deps.edn maybe #2019-08-07 22:02:10 hiredman on pardon me, that does come from tools deps, I wonder why it is categorized as a syntax error #2019-08-07 22:04:07 hkupty I assume it's complaining about the dependency declaration syntax, as it expects ns:symbol:jar:version where version is a string, but instead it gets a list. #2019-08-07 22:05:14 alexmiller @U0NCTKEV8 the syntax thing is a bug, I have a patch pending for that #2019-08-07 22:05:22 hiredman clj -Sdeps "{:deps {io.etcd/jetcd-core {:mvn/version \"0.3.0\"}}}" might give you a clearer error #2019-08-07 22:06:08 hiredman it looks like something depends on an artifact that is not in maven central #2019-08-07 22:06:27 alexmiller if you're calling this with tools.deps directly, you need to pass it the repos #2019-08-07 22:06:39 alexmiller clj provides those, but tools.deps does not (automatically) #2019-08-07 22:08:40 hkupty I'll check that. However, the thing is I can do
(deps/resolve-deps {:deps {'io.grpc/grpc-core {:mvn/version "1.17.1"}}} {}) #2019-08-07 22:09:09 hiredman but that is a different thing #2019-08-07 22:09:13 alexmiller b/c you have it locally #2019-08-07 22:09:13 hkupty But I can't do the same for io.etcd/jetcd-core, which fails on this very same dependency. #2019-08-07 22:11:28 hiredman it does not for me, for me it fails to find io.netty:netty-codec-http2:jar:[4.1.30.Final] in central #2019-08-07 22:12:03 hkupty Right. I got a Syntax error (ExceptionInfo) compiling at (cli/ops.clj:5:1).\nCould not find artifact io.grpc:grpc-core:jar:1.17.1 after moving my ~/.m2 to ~/.m2-bkp`. #2019-08-07 22:12:21 hkupty So I'll make sure to add the extra repositories #2019-08-07 22:13:08 hiredman it would be great if you could try the command line call to clj #2019-08-07 22:14:12 hiredman it looks like the place in deps where that exception is thrown is if the dep is either a version range or an alias like LATEST or RELEASE #2019-08-07 22:14:49 hiredman so my guess something in jetcd is changing how maven attempts to resolve an alias or a range in io.grpc's dependencies #2019-08-07 22:15:10 hkupty I could make it work by adding "central" {...} to :mvn/repos #2019-08-07 22:15:23 hkupty that solves everything #2019-08-07 22:16:10 hiredman hrmm, I am still getting errors, and it has changed to different artifacts not being found in central now 😞 #2019-08-07 22:16:25 hkupty 😱 #2019-08-07 22:16:59 hiredman but I am not using jetcd-core so no big deal #2019-08-07 22:18:43 hkupty 😅 Thanks to you both for all the help and guidance 🙂 #2019-08-07 22:28:59 seancorfield In light of continued complaints about the use of "RELEASE" in projects created by clj-new ... seancorfield/clj-new {:mvn/version "0.7.7"} pins these to 1.10.1 (for Clojure) and 0.10.0-RC1 (for test.check). You're welcome 🙂 #2019-08-07 22:29:57 seancorfield Bear in mind that lein new, boot new, and clj-new all look for templates themselves (on Clojars/Central) using "RELEASE" under the hood unless you specify a template version to use (which almost no one does). #2019-08-07 22:30:06 ghadi thanks @seancorfield #2019-08-07 23:14:27 andy.fingerhut This is not a critical question in any way, but was curious whether folks had any recommendations on creating a deps.edn file for a library that has Clojure-specific and ClojureScript-specific flavors in the same git repo, e.g. core.rrb-vector is the one I have in mind. One thought: create a deps.edn that has almost 0 dependencies when no aliases are specified, and has a :clj alias for Clojure dependencies/paths, and a :cljs alias for ClojureScript dependencies/paths. Second thought: Two separate deps.edn files, except there is only one root directory file named deps.edn available. #2019-08-07 23:15:08 andy.fingerhut Third thought: two separate git repos, one for each variant? #2019-08-08 02:08:47 alexmiller Just fyi on kind of a tangent, core.rrb-vector is a Clojure contrib and the only build officially supported for those for build and deploy is the pom.xml - that has to be the primary #2019-08-08 02:09:05 andy.fingerhut Understood. #2019-08-08 02:09:08 alexmiller it's ok for it to also have a deps.edn #2019-08-08 02:09:25 alexmiller and the tool for variant classpaths is aliases #2019-08-08 02:09:30 andy.fingerhut But the pom.xml is a no-op for ClojureScript, isn't it? #2019-08-08 02:10:01 andy.fingerhut oh, maybe not. #2019-08-08 02:10:03 alexmiller we have other contrib libs that build a single jar usable from either #2019-08-08 02:10:11 alexmiller data.xml, tools.reader, etc #2019-08-08 02:10:26 andy.fingerhut I guess the pom.xml controls what ClojureScript source files are bundled up in there. #2019-08-08 02:10:52 alexmiller yes #2019-08-08 02:10:56 seancorfield (tools.cli is pure .cljc so it may not be a good example for you) #2019-08-08 02:11:28 alexmiller the trickiest part of clj/cljs contribs is testing and probably data.xml is the most advanced example out there #2019-08-08 02:14:04 andy.fingerhut I will take a look at data.xml more carefully. I know pretty much all contribs do automated build and at least a subset of their tests automated with results on http://build.clojure.org. Do you know if any of them build and test ClojureScript code? #2019-08-08 02:17:05 andy.fingerhut Ah, I see in the console log for data.xml that it does run some cljs tests using nashorn JavaScript runtime. #2019-08-08 02:20:23 seancorfield I should have a look at that too since the Maven build for tools.cli only does clj testing, and I do cljs testing manually via deps.edn... #2019-08-08 02:22:47 alexmiller those are the main ones I can think of that do cljs testing #2019-08-08 02:23:05 alexmiller "building" is really just including the cljs in the jar so not really a big deal, a bunch of contribs do that #2019-08-08 02:23:36 alexmiller in a lib like core.async there is a manual cljs test process that we run by hand, but it's not part of ci #2019-08-08 02:26:08 andy.fingerhut So that is a great lead-in to what the pom.xml should ideally do (i.e. automate ClojureScript as well as Clojure tests), and I can look into that. Definitely nicer with a working example to start from. #2019-08-08 02:26:49 andy.fingerhut As far as deps.edn file goes, would it be bad to have one, if its contents conflict with what pom.xml says? #2019-08-08 02:27:13 alexmiller no, not bad #2019-08-08 02:28:58 alexmiller many of the contribs also have project.clj files #2019-08-08 02:29:55 andy.fingerhut deps.edn in a Maven-deployed JAR file should either (a) not be in the JAR, or (b) ignored by tools.deps, Maven, etc. even if one is in there? #2019-08-08 02:31:48 alexmiller I don't think it would be in the jar file #2019-08-08 02:31:54 alexmiller and it would be ignored by Maven #2019-08-08 02:32:22 alexmiller there are plenty of examples to check -a bunch have deps.edn files #2019-08-08 02:32:41 alexmiller tools.deps won't ignore it but presumably that's the point #2019-08-08 02:32:55 alexmiller if this gets too into the weeds on contrib, I would move it to #clojure-dev #2019-08-08 02:33:58 andy.fingerhut Understood. If I haven't exhausted you with questions yet, is it short to explain what happens when tools.deps follows a dependency to a git repo, and that git repo has both pom.xml and deps.edn files? #2019-08-08 02:34:37 andy.fingerhut If that is involved, no worries, I can wait. #2019-08-08 02:34:49 andy.fingerhut (or experiment) #2019-08-08 02:37:27 alexmiller deps.edn is first #2019-08-08 02:38:24 alexmiller that's why I said it matters to tools.dep #2019-08-08 02:39:23 andy.fingerhut thx. Keeping some notes separate from here for future reference. #2019-08-08 03:03:32 seancorfield @andy.fingerhut When building JARs, you're mostly just going to get src and the META-INF stuff (which includes a pom.xml file) but nothing else from the repo (so no deps.edn file). #2019-08-08 03:04:06 seancorfield When you depend on a JAR, the transitive dependencies are figured out from that embedded pom.xml file (as I found out recently with depstar!). #2019-08-08 03:05:31 seancorfield When you depend on a git or local repo, tools.deps will use deps.edn if present else pom.xml. If neither are present, you can tell tools.deps to "assume" deps.edn (and it treats the absence as an empty {}). #2019-08-08 03:06:04 seancorfield :deps/manifest :deps #2019-08-08 03:13:11 andy.fingerhut thx! #2019-08-08 15:05:46 hkupty Hey, when using tools.deps on a git dependency (which is a tools.deps project itself as well), can I load an alias of this dependency?
The use-case is: I want to dynamically load a project using tools.deps+dynpath (up to this point I got it running), access its test/ folder and run the tests.
No specific project yet, just trying some ideas. #2019-08-08 15:11:50 alexmiller no, not supported #2019-08-08 15:11:54 alexmiller but something under consideration #2019-08-08 15:13:01 hkupty I think that could be solved by adding paths and deps with resolve-deps and make-classpath then. #2019-08-08 15:16:10 hkupty If I got it right, then calling (make-classpath {...} ["test"] nil) could allow that, right? #2019-08-08 15:20:53 alexmiller need more words to understand what you mean #2019-08-08 15:21:06 alexmiller what is "that" and "it"? #2019-08-08 16:01:04 hkupty My intent of being able to run the tests from a dependency I pull dynamically as explained in the use case above. #2019-08-08 16:09:20 alexmiller then, yes that should work #2019-08-08 16:09:35 alexmiller you probably want to also include the source paths though I would assume #2019-08-08 16:09:52 alexmiller like ["src" "test"] #2019-08-08 16:12:57 hkupty Good. Thanks for the help. #2019-08-08 18:06:02 jrwdunham Can someone here help me to understand how I can rebuild/re-install a local dependency without quitting and restarting Spacemacs/Cider? I was originally declaring a Git dependency and then I changed my deps.edn file so that the dependency was to a local path. That is, my deps.edn started out as {:deps github-myorg/mylib {:git/url "" :sha "6c9cbfa1b1721b3d7789e6ff65a3c01065fd6c54"}} and changed to {:deps mylib {:local/root "/path/to/root/of/mylib"}}. #2019-08-08 18:58:38 alexmiller you'll have to restart your repl to pick that up #2019-08-08 19:06:28 jrwdunham thanks @alexmiller. I was thinking that would be necessary, but was hoping not #2019-08-08 23:12:14 kenny I have a project that depends on a library A via :local/root. Library A depends on com.google.cloud/google-cloud-monitoring. google-cloud-monitoring depends on commons-logging. In library A, I have :exclusions [commons-logging] set on com.google.cloud/google-cloud-monitoring. When I run clj -Stree in my top level project, I still get commons-logging/commons-logging in the tree. If I run clj -Stree in library A, I also get commons-logging. Any idea what is going on here? #2019-08-08 23:12:42 kenny Global exclusions would solve this btw ^ #2019-08-08 23:14:13 seancorfield @kenny commons-logging is used by a lot of things -- perhaps something else is bringing it in? #2019-08-08 23:14:31 kenny Perhaps. But it is shown under com.google.cloud/google-cloud-monitoring #2019-08-08 23:14:31 alexmiller where do you see it in the tree? #2019-08-08 23:15:13 alexmiller did you try :exclusions [commons-logging/commons-logging] ? #2019-08-08 23:15:18 kenny Well, several layers deep in there #2019-08-08 23:15:27 kenny No, I can though. I thought they'd be the same. #2019-08-08 23:15:36 alexmiller I wonder if there's a non-canonicalized matching canonicalized somewhere #2019-08-08 23:15:45 kenny com.google.cloud/google-cloud-monitoring 1.78.0
javax.annotation/javax.annotation-api 1.3.2
com.google.cloud/google-cloud-core-grpc 1.78.0
com.google.auth/google-auth-library-credentials 0.16.1
io.grpc/grpc-protobuf 1.21.0
io.grpc/grpc-protobuf-lite 1.21.0
com.google.api/gax-grpc 1.45.0
io.grpc/grpc-alts 1.21.0
org.apache.commons/commons-lang3 3.5
io.grpc/grpc-grpclb 1.21.0
com.google.protobuf/protobuf-java 3.7.1
io.grpc/grpc-stub 1.21.0
com.google.api.grpc/proto-google-cloud-monitoring-v3 1.60.0
com.google.cloud/google-cloud-core 1.78.0
com.google.api.grpc/proto-google-iam-v1 0.12.0
com.google.protobuf/protobuf-java-util 3.7.1
com.google.api/api-common 1.8.1
com.google.http-client/google-http-client 1.30.1
com.google.j2objc/j2objc-annotations 1.3
io.opencensus/opencensus-contrib-http-util 0.21.0
org.apache.httpcomponents/httpclient 4.5.8
org.apache.httpcomponents/httpcore 4.4.11
commons-logging/commons-logging 1.2
#2019-08-08 23:16:09 alexmiller if you can, I'd try changing the exclusion #2019-08-08 23:16:13 kenny Commenting google-cloud-monitoring shows another dep bringing it in. I guess I can follow the trail. #2019-08-08 23:17:41 kenny commons-logging/commons-logging does it #2019-08-08 23:18:16 kenny Gosh, I wonder how many other places have an exclusion that isn't qualified and it should be 🤔 #2019-08-08 23:19:24 kenny Wait, now commons-logging is showing up in another location in the tree. #2019-08-08 23:20:11 alexmiller yeah, the code doesn't canonicalize the exclusions #2019-08-08 23:21:45 kenny Changing the :exclusions in a :local/root dep does not invalidate the clj cache. #2019-08-08 23:38:50 alexmiller Yeah, that’s a known issue #2019-08-08 23:39:32 alexmiller -Sforce #2019-08-08 23:42:30 alexmiller I have a plan to fix it but it’s a pain #2019-08-08 23:43:12 kenny Gotcha, ok. Out of curiosity, would it be possible to resolve deps in parallel? #2019-08-08 23:51:56 alexmiller Yes, have thought about it some #2019-08-08 23:53:06 alexmiller Generally you have most stuff in local repo already so I don’t find it critical #2019-08-09 00:20:19 kenny The piece that is more interesting to me is shortening the time it takes to get a classpath. We have some services where it takes nearly a minute to get a classpath. #2019-08-09 02:22:09 alexmiller why? #2019-08-09 02:23:25 alexmiller are you i/o bound (and if so, why?) or is the tree huge? or it something in resolution (cpu bound)? #2019-08-09 02:44:13 alexmiller a minute seems super long so would like to learn more about why #2019-08-09 02:45:46 alexmiller also, curious what version of clj you're on? Just reading the code, it seems like the exclusions should get canonicalized in this scenario, at least in current code #2019-08-09 02:47:46 alexmiller oh, I see why #2019-08-09 02:56:17 andy.fingerhut Are there transitive dependency reachability checks done using exponential time algorithms? 🙂 #2019-08-09 02:57:06 andy.fingerhut Probably a completely irrelevant remark -- I just recall finding and fixing such a thing in tools.namespace years back. #2019-08-09 03:03:15 alexmiller it's designed to be iterative and should be a single pass over the tree #2019-08-09 03:05:02 alexmiller but problems like missing a cycle could cause issues #2019-08-09 03:07:18 alexmiller @kenny I found and fixed the bug with exclusion canonicalization, really a problem with exclusions in any deps.edn, not just transitive. very good catch! #2019-08-09 03:07:33 alexmiller not going to release it right now but will be in whatever the next release is #2019-08-09 03:25:03 seancorfield I looked at our deps and we certainly have a mix of bare and canonicalized exclusions so it'll be interesting to see what affect that has on us 🙂 #2019-08-09 03:47:20 alexmiller my read of current release is that all bare exclusions are being ignored #2019-08-09 03:48:21 seancorfield Thanks. I'll try to qualify all of ours and see if anything falls out... #2019-08-09 03:51:01 alexmiller might want to do an -Stree before/after #2019-08-09 14:02:59 kenny @alexmiller Thanks for the fix!
> are you i/o bound (and if so, why?) or is the tree huge? or it something in resolution (cpu bound)?
Just on a regular computer w/ ssd so not i/o bound. The tree is pretty big but I'd expect a tree this size with any enterprise product. Could be something in resolution that is slow, don't think it's cpu. #2019-08-09 14:05:29 alexmiller I mean i/o bound in talking to the network to download jars #2019-08-09 14:05:42 alexmiller not filesystem #2019-08-09 14:05:58 kenny Oh, no. I think everything is already on disk #2019-08-09 14:06:24 alexmiller like during that minute, could you grab periodic stack traces, either with ctrl-\ or with jstack, and look at the top of the stack? #2019-08-09 14:06:51 kenny Sure #2019-08-09 14:07:22 alexmiller so parallel downloads would not help you at all if you're not downloading anything #2019-08-09 14:07:44 kenny Right #2019-08-09 14:08:16 alexmiller another debug thing to do is -Sdeps '{:aliases {:v {:verbose true}}}' -A:v #2019-08-09 14:08:46 kenny BTW, the slowness is a "full" refresh, not using the cache. #2019-08-09 14:08:59 kenny e.g. adding/changing/removing a dep #2019-08-09 14:08:59 alexmiller if you see pauses in there, that would be suspicious, but otherwise, if you just have a big trace, I'd be interested in seeing that, could dm it to me #2019-08-09 14:09:30 alexmiller not using which cache? classpath cache? m2 local repo? gitlibs cache? #2019-08-09 14:10:07 kenny Not sure. Changing certain deps.edn takes a long time. #2019-08-09 14:10:44 alexmiller so you're not actively clearing the m2 repo or anything #2019-08-09 14:10:49 kenny Oh, no. #2019-08-09 14:10:53 kenny Same for gitlibs and the rest #2019-08-09 14:10:53 alexmiller so really classpath cache #2019-08-09 14:11:04 alexmiller that's the only thing stale when you change deps.edn #2019-08-09 14:12:03 alexmiller and you're on latest clj? #2019-08-09 14:12:21 alexmiller there were some cycle detection issues that were fixed months ago #2019-08-09 14:12:50 alexmiller clj -Sverbose for version #2019-08-09 14:12:54 kenny This is from -Sdescribe :version "1.10.1.466" #2019-08-09 14:13:00 alexmiller yeah, that's latest #2019-08-09 14:13:12 alexmiller well, I'd love to take a look #2019-08-09 14:16:47 kenny For starters, these messages have reappeared and they take a decent chunk of time:
Downloading: io/grpc/grpc-api/maven-metadata.xml from
Downloading: io/grpc/grpc-core/maven-metadata.xml from
Downloading: io/grpc/grpc-netty-shaded/maven-metadata.xml from
@seancorfield had noticed that a pom from one of my deps (com.google.cloud/google-cloud-monitoring "1.78.0") used a RELEASE version. He suggested explicitly specifying the deps mentioned there. I have done that and they still appear. #2019-08-09 14:17:47 alexmiller yeah, that's actually an s3 wagon issue, upstream from tools.deps #2019-08-09 14:18:01 kenny He also mentioned it may have something to do with not correctly resolving deps from a parent pom. #2019-08-09 14:18:14 kenny Oh. Well that could easily shave ~15s off the time. #2019-08-09 14:18:52 kenny Those deps are coming from google-cloud-monitoring not Datomic #2019-08-09 14:26:09 alexmiller I don't have enough info to debug this, would be useful to see deps.edn and the verbose trace above #2019-08-09 14:28:15 kenny Sure. I can send that over. I'll see if I can create a smaller deps.edn first. #2019-08-09 14:29:54 kenny Just this will do it:
{:deps {com.google.cloud/google-cloud-monitoring {:mvn/version "1.78.0"}}
:mvn/repos {"datomic-cloud" {:url ""}}}
#2019-08-09 14:31:34 alexmiller this is starting to ring a bell #2019-08-09 14:32:16 alexmiller there's a loop in these maven deps iirc #2019-08-09 14:37:11 kenny Adding time to the clj calls show I drastically overestimated the time it takes:
real 0m23.962s
user 0m46.126s
sys 0m1.158s
This certainly feels like an eternity when needing to do that many times a day. A big portion of that time is the "Downloading: ..." thing. It would be a huge productivity boost to get that under 5s. #2019-08-09 14:38:03 ghadi why does repeated downloading happen in your environment? #2019-08-09 14:38:16 alexmiller that's an issue with the s3 wagon I think #2019-08-09 14:38:24 ghadi (jumping into this conversation without reading the backscroll) #2019-08-09 14:38:35 kenny Does it not do that for you? #2019-08-09 14:38:49 alexmiller I can repro it #2019-08-09 14:39:56 alexmiller without the datomic repo in the mix, it's about 5 seconds to build a classpath for that #2019-08-09 14:40:22 kenny That s3-wagon thing has always been a nightmare for me. I remember always hitting issues with it back when we used an s3 maven repo. Perhaps a good use case for aws-api? 🙂 #2019-08-09 14:40:41 alexmiller with it, I see about 8-9 seconds #2019-08-09 14:41:09 kenny Yeah, I'd be curious what one of our large apps takes without the downloading thing. #2019-08-09 14:41:16 alexmiller time clj -Spath -Sforce #2019-08-09 14:41:37 kenny That will still have the downloading issue. #2019-08-09 14:42:02 alexmiller yeah, that's the idea :) #2019-08-09 14:42:17 kenny Oh. I already sent that above haha #2019-08-09 18:31:57 ikitommi any success stories of multi-module/mono-repo library setups with deps? have some working lein projects for that, but have now a deps project that needs to be split into parts. #2019-08-09 18:46:53 kenny @ikitommi We switched to a monorepo a month or so ago. All internal libraries are :local/root. Makes working in the REPL great. There's a few kinks with our setup though:
- We use CircleCI for CI/CD. There's no support for monorepos with CircleCI. This leads to longer build times - every project runs through its test steps with every push. We recently switched to their new unlimited parallelism plan which has been quite helpful in getting CI time down.
- CI configuration was moved into a set of clojure files because it became far too tedious messing around with YAML with the number of projects we have. This does, unfortunately, mean that CI config needs to be manually generated with a command every time you change the CI clojure files.
- We have a small service diff library that detects when a particular service's code, deps.edn, or :local/root deps have changed. That ensures a service won't get deployed on every push.
- We don't have a great way to have a common deps.edn across all projects. This would be quite useful for things like: global exclusions (when supported), overriding certain library versions, common aliases, etc. Ideally there'd be some way to just pass in N number of deps.edn files to clj and have it merge those in. I use Cursive so Cursive would also need to have some way to select which deps.edn files to use.
- We don't have a good way to run commands across all projects or only in a certain project. For example, I'd like to be able to do something like: monorepo my-service uberjar.
- Different libraries & services run tests with a different set of aliases. Every time we want to run the tests for a project, you need to go to the projects README (or check the CI config) and determine which aliases to use to run the tests. Either the aforementioned "command runner" or a way to combine aliases somehow would make this much better.
Overall, this workflow is far better than having individual repos and constantly needing to restart the REPL when working across projects. #2019-08-09 19:04:24 ikitommi Thanks @kenny! was hoping for the monorepo kinda script, too lazy to start cooking up own tools right now. In my case, it's a library, going to be split into set of libraries, so would be easy to have same aliases for all. A sample repo would be super awesome. #2019-08-09 19:11:53 dominicm @ikitommi you might want to look at edge which does this #2019-08-09 19:12:20 dominicm Also Sean has talked a lot about their setup at world singles #2019-08-09 19:17:11 seancorfield We have a monorepo with maybe two dozen subprojects, and 90k lines of Clojure. #2019-08-09 19:18:33 seancorfield The key thing we did was to have a primary deps.edn in a folder and point to that via CLJ_CONFIG (so it "replaces" the user-level deps.edn) and then each subproject has a deps.edn. #2019-08-09 19:19:08 seancorfield We use :override-deps in the primary file to "pin" versions of libs across the whole repo as needed, as well as provide all the common tooling via aliases. #2019-08-09 19:19:55 seancorfield The only "tooling" we've built on top of this is a small shell script that can execute multiple clojure commands and knows how to navigate to subprojects when running series of commands. #2019-08-09 19:21:05 seancorfield Like @kenny we use :local/root deps for cross-module deps -- and we have an everything subproject that we can build the deps.edn into from across the monorepo and that's where we usually start our REPL/REBL from. #2019-08-09 19:22:54 kenny Hmm that's a good idea! Cursive doesn't support CLJ_CONFIG unfortunately. Creating a everything project and starting a nrepl from the command line could solve that problem though! Generally it makes sense to have everything on the classpath while dev'ing. #2019-08-09 19:23:57 kenny How does your script know which aliases to use for each project's tests? #2019-08-09 19:34:08 seancorfield build set:of:aliases subproject is our shell script. But it can take multiple pairs of aliases/subprojects. #2019-08-09 19:34:57 seancorfield and if we need arguments, we can use [ ] to wrap them, so build uberjar api [ run ci-ftp api #2019-08-09 19:43:01 kenny How do expose build? Is it a script at the root of the repo? Do you have devs add to PATH? #2019-08-09 19:55:59 seancorfield We have a <repo>/build/bin folder containing scripts. Devs can either add it to their path or just run the scripts directly. #2019-08-09 19:56:44 seancorfield I mostly work in the build folder but I have build/bin/build symlinked into my ~/bin folder for convenience. Other stuff I run with ./bin/<script> #2019-08-09 19:57:47 seancorfield Between docker compose and two git clone commands, a dev can be set up "immediately" (assuming they have an OpenJDK8 installed). #2019-08-09 19:58:09 seancorfield If they need to work on our legacy apps, there's one more git clone to run. #2019-08-09 19:59:04 seancorfield (one of those repos is for semi-static tooling, which is where we run docker compose -- for Redis, Elastic Search, Percona/MySQL, and a custom search engine we use) #2019-08-09 20:17:07 seancorfield Interesting little trick! #2019-08-09 22:22:18 kenny Does CLJ_CONFIG have to be an absolute path? #2019-08-09 22:23:31 kenny I think it doesn't. I thought it wasn't working for a sec. #2019-08-09 22:23:32 seancorfield Nope. We use "../versions" in our script. #2019-08-09 22:23:55 kenny How do you deal with not knowing where the build script is run from? #2019-08-09 22:24:23 seancorfield CLJ_CONFIG=../versions clojure -A:defaults:<task> <args>
:defaults pulls in all the overrides etc from versions/deps.edn #2019-08-09 22:24:58 kenny For example: if I have a build/bin/build and I run it from build/bin, I need to know to set CLJ_CONFIG to ../../versions. If I run it from build, I need to set CLJ_CONFIG to ../versions. #2019-08-09 22:25:48 seancorfield dirname $0 #2019-08-09 22:26:05 seancorfield Then you can work relative to that. #2019-08-09 22:26:52 kenny Oooo, I didn't know about $0 #2019-08-09 22:27:24 seancorfield (although we assume certain filesystem paths are the same on all dev/test/prod images so some of our scripts take advantage of that -- and devs just add a symlink to wherever they decided to put stuff) #2019-08-09 22:50:48 kenny Given you symlink build, dirname $0 will return a path to wherever the symlink is. How do you deal with that? #2019-08-09 22:51:48 kenny Perhaps readlink? #2019-08-09 23:20:40 seancorfield Yes, you can use readlink $0 to get the actual file location (it exits with a non-zero status if the argument is not a symlink). #2019-08-09 23:21:55 kenny You must then also deal with Linux/Mac platform differences with readlink 😵 #2019-08-09 23:22:41 seancorfield Like I said above, we also assume certainly filesystem paths to make our lives easier 🙂 #2019-08-09 23:23:41 seancorfield (partly because we have a shell script in the main repo that a new dev can download and it does most of the env setup for them, including git cloneing repos to specific places and setting up symlinks and handling the initial Mac/Linux differences) #2019-08-09 23:24:30 seancorfield But there are plenty of ways to skin that particular kitty. #2019-08-09 22:59:19 hkupty What is the difference between:
• (resolve-deps {:paths [p1 p2] ...}),
• (make-classpath ... [p1 p2]) and
• (make-classpath .. nil {:extra-paths [p1 p2]})
All three seem to produce the same result, which is to add the local project's paths p1 and p2 to the classpath.
To further exemplify my question from yesterday, I'm looking for something on the lines of:
{:deps {some/dep {:git/url ... :paths [p1 p2]}}}
Is the example above feasible? #2019-08-09 23:31:39 alexmiller Those 3 produce the same result but are semantically different #2019-08-09 23:32:29 alexmiller What’s your goal? #2019-08-09 23:33:33 hkupty I want to add a folder (for example 'test') from a dependency I loaded w/ tools.deps to my currently running project's classpath. #2019-08-09 23:34:39 hkupty That folder is not declared on the main {:paths [...]} clause on dependency's deps.edn for obvious reasons, it is not the mainline for that project. #2019-08-09 23:36:41 hkupty However, as I'm trying to build something like a buildscript CLI interface for a group of projects, I want to dynamically load them using tools.deps and run their tests, or whatever else I might want to do with them, given that not only I can load their dependencies but also add folders (or other aliases from that deps.edn) into this buildscript CLI classpath. #2019-08-09 23:46:45 hkupty There's a hacky way for me to work around that, which is to hijack the result of (result-deps) through some (update-in (result-deps ...) [dep :paths] conj '/my/hand-crafted/path/p2') before (make-classpath ...). That seems too hacky for me, but I can do that if tools.deps doesn't want to explore further the project structure of a dependency. #2019-08-09 23:48:17 alexmiller If you’re calling tools.deps programmatically then you’re already in the machine - do whatever you want with the intermediate results #2019-08-09 23:52:30 kenny @ikitommi I added a build script to our repo that is similar to the one @seancorfield described. Here's what I ended up with. Our repo is structured like with all projects under projects and this script located at bin/build. #2019-08-10 00:03:07 seancorfield I should probably see if I can synthesize a generic version of our build script and release it as open source... #2019-08-10 00:04:53 kenny That would be super awesome 🙂 I just realized mine doesn't support aliases very nicely... #2019-08-10 00:09:55 seancorfield We have a whole bunch of WS-specific alias expansions in it right now. build run thing will expand into whatever thing means in our project. So build run database-setup expands to build [ migrate migration photos dev ] which runs the :migrate alias in the migration subproject and then runs the :photos alias in the dev subproject. #2019-08-10 00:12:08 seancorfield We also have an expand-deps.sh script that looks for :local/root deps in a subproject's deps.edn file and recursively expands the list of subprojects to include all transitive subprojects that are depended on -- so we can say build test-all api and it will expand to build tests environment datamapper worldsingles ... api and that in turn expands to running build test <subproject> for each subproject in turn 🙂 #2019-08-10 00:13:17 seancorfield prev=""
curr="
where $clojure is the monorepo root. #2019-08-10 00:15:21 seancorfield All told, including the WS-specific task alias expansion and the 70 lines of help output, we're talking 350 lines of bash. #2019-08-10 00:19:44 kenny Hmm, that's a lot of bash haha. I just about hit my tolerance with the wee 16 lines above 😛 #2019-08-10 00:24:52 seancorfield I've created some truly monstrous bash-based systems in my career... 1,000's of lines long. I probably wrote my first *nix shell script back in '79 tho'... neckbeard #2019-08-10 00:27:54 kenny Writing a small script to do the necessary parsing in plain ol' Clojure only adds 1s overhead 🙂 #2019-08-10 00:37:30 seancorfield True, I could rewrite all this in Clojure (and shell out to clojure as needed for the repeated invocations). I could make all the WS-specific stuff data driven and external too. Maybe one day 🤔 #2019-08-10 00:59:01 alexmiller @kenny I made some progress on your perf issue today #2019-08-10 00:59:50 alexmiller There’s some repository cache stuff built into Maven that we weren’t using so I’ve changed that and it makes a big difference. #2019-08-10 01:01:01 alexmiller There’s still something I don’t understand about why you’re getting that s3 maven download every time, but it’s now only doing it once at least. #2019-08-10 01:08:27 kenny Awesome! Is there a way for me to test it out before a release? #2019-08-10 01:13:07 seancorfield @kenny Looking at the clojure script, I think the answer is "not easily". #2019-08-10 01:16:05 seancorfield You'd have to build tools.deps JAR yourself and overwrite the version in the clojure install or hack the clojure script to point to that JAR elsewhere #2019-08-10 01:21:44 kenny That doesn’t sound that bad. Perhaps writing bash makes tasks like that sound easy haha #2019-08-10 01:32:50 alexmiller You don’t even need to build it #2019-08-10 01:33:03 alexmiller Just clone the repo #2019-08-10 01:35:05 alexmiller Then modify your clojure script to put the tools.deps.alpha/src/main/clojure dir at the front of the tools_cp var #2019-08-10 01:38:27 alexmiller Or you can just wait a day or two for me to test and release it :) #2019-08-10 06:05:32 dominicm There's one downside to not building it since the removal of the system deps.edn, you end up with templated version strings! So you need a hard clojure dependency. #2019-08-10 12:39:29 alexmiller Ah, true. Well you can build and use the target output, that should work. #2019-08-12 04:38:22 onetom is there a definitive guide on how to combine some java sources into your deps.edn clojure project?
like which directories should i put my java sources?
when and how to run the compilation into .class files?
where should i put the resulting class files, so i can import them from clojure?
things like that. #2019-08-12 04:39:05 onetom i have some protobuf .java files from a decompiled android app and i would like to use it via clojure repl... #2019-08-12 04:47:00 onetom i do need to access the same maven deps from java which i specified in deps.edn though...
i know from the command line i can just do java -cp $(clj -Spaths)...,
but do i need something like this for javac compilation too?
( plus i what i really don't have an idea about is how to make symbols resolve in cursive in java sources; but that i can ask on #cursive ) #2019-08-12 04:48:15 cfleming @onetom For source paths, you can just use :extra-paths. Once you have that set up, Cursive will just work. #2019-08-12 04:49:02 onetom can i just mix the java source files into the same src/ directory as the clojure files then? #2019-08-12 04:49:29 cfleming There’s no lein javac equivalent in deps. You’ll need to set up aliases for compilation and runtime - compilation will have your Java sources on the classpath, and runtime will have the resulting class files. #2019-08-12 04:49:54 cfleming You can mix them, yes, but your Java source files will be on the runtime classpath then. #2019-08-12 04:50:14 cfleming I’d keep them separate if I were you, it’s trivial to do so. #2019-08-12 04:50:52 cfleming You’ll also need a run alias to do the actual compilation, but I’ve not investigated that. #2019-08-12 04:51:01 onetom ah, i see.
so is there any templates out there which showcase some "recommended/reasonable" setup? #2019-08-12 04:51:19 cfleming There’s no concept of automatically compiling before run with deps, so you’ll need to manually recompile if anything has changed. #2019-08-12 04:52:06 cfleming You could set that up in IntelliJ and just let IntelliJ compile for you, depending on how official or serious your project is #2019-08-12 04:52:11 onetom the built-in Build feature would be intuitive to be used for this purpose #2019-08-12 04:52:39 cfleming Yes, you can do that. You could just add a Build Project before-run step to your run configuration. #2019-08-12 04:53:44 cfleming #2019-08-12 04:54:30 cfleming So to set that up set up Java compilation as normal, and add an alias to your project which adds the compile output path to your classpath. #2019-08-12 04:54:57 onetom thanks a lot! i will try it.
i hope i don't have to pull in some build tool for this simple java compilation 🙂 #2019-08-12 04:56:25 cfleming Then in your run config, run with deps and use your runtime alias to add the compilation output to the classpath. #2019-08-12 04:56:32 cfleming And you should be good to go 🙂 #2019-08-12 04:57:34 onetom i only understood half of it, but haven't seen your deps.html documentation yet, so it will be clear afterwards, im sure #2019-08-12 04:57:54 cfleming Ok, ask over in #cursive if you have problems, it’s more of a Cursive thing than a deps thing. #2019-08-12 04:58:26 onetom btw, i've just upgraded to IntelliJ 2019.2.1 Preview last night and indeed Cursive deps support works without issues with it so far! #2019-08-12 04:59:24 cfleming Yes, they fixed a few problems there, but there are still more. I’m actually in the process of migrating the deps support off the IntelliJ API I was using, and instead basing it on the lein support, so I’ll be insulated from all their platform issues. #2019-08-12 11:01:53 kvlt I've noticed since moving to deps.edn my log lines no longer show up in my emacs repl buffer. Anybody know why this is? #2019-08-12 11:05:32 kszabo > And there have been some complaints that tools.deps includes slf4j-nop as a dep. slf4j is a pluggable logging framework and the nop logger is a “don’t log” plugin, which takes away the choice for library consumers to plug as they wish. tools.deps will remove that dependency but it will still be included in the clj installation. #2019-08-12 11:14:05 kvlt Thank you! So I should just be able to add the dependency and it should log again? #2019-08-12 13:21:17 alexmiller just using clojure to build a classpath should not affect the clojure program you run with the classpath after that (as tools.deps is not included in that). #2019-08-12 13:21:48 alexmiller and that's how I'm interpreting your problem statement at the top #2019-08-12 13:22:55 alexmiller it's possible that cider is also relying on tools.deps for something and that is affecting your program somehow, but I don't know enough about how or whether cider uses tools.deps to know if that could be related #2019-08-12 13:24:32 alexmiller I would start by opening an issue or asking a question in #cider before jumping to the conclusion that this is related #2019-08-14 11:02:49 unbalanced So I'm looking for a command line solution to handling generic tooling tasks such as "bump version number", or "modify deps.edn to add something to :deps" ... do we just write our own scripts to do that sort of thing or is there a canonical solution? #2019-08-14 11:06:02 dominicm (and lots of other useful tools there too) #2019-08-14 11:06:26 unbalanced Also I'm still really curious what the heck a build.clj file is, such this:
(require '[cljs.build.api :as b])
(b/build "src"
{:output-dir "out"
:output-to "out/main.js"
:optimizations :none
:main 'example.core
:install-deps true
:npm-deps {:react "15.6.1"
:react-dom "15.6.1"}})
reference here #2019-08-14 11:06:51 dominicm It's nothing special. It's just a clojure program that can be run. #2019-08-14 11:07:30 unbalanced Maybe an intermediate step on the way to tools-deps? #2019-08-14 11:07:44 dominicm No. tools-deps doesn't care abut build tools at all. #2019-08-14 11:08:17 dominicm To run their example, back in 2017 you did this: java -cp ls ../../lib/*.jar | paste -sd ":" -`:../../src/main/cljs:../../src/main/clojure:src clojure.main build.clj`
with tools-deps you can do this though: clojure build.clj #2019-08-14 11:11:29 unbalanced 2017 was rough 😬 #2019-08-14 11:12:01 dominicm probably why so many used lein #2019-08-14 11:19:02 unbalanced Is there a place in deps.edn where we put license/version info or does that go somewhere else now? #2019-08-14 11:20:46 dominicm somewhere else. pom.xml like before would be fine I think. #2019-08-14 11:22:42 unbalanced 👍 #2019-08-14 15:31:39 dominicm @alexmiller in tools.deps.alpha.reader, is it worth having the user-deps locator as a separate function? In pack the issue has been raised to include part of the user deps.edn which seems acceptable. That part is just for including maven repo settings which are user-local. #2019-08-14 15:33:25 alexmiller Sure #2019-08-14 15:35:16 dominicm will open a jira #2019-08-15 06:38:27 murtaza52 is there any tooling for auto completion built on top of the tools.deps repl (started using the clj command) ? #2019-08-15 06:47:26 seancorfield The Rebel Read line tool might be what you're looking for. There is an alias for it in my dot-clojure repo deps.edn #2019-08-16 16:18:30 zane There is also lots of editor-specific tooling for auto-completion that plays well with tools.deps-based projects. #2019-08-18 08:47:13 murtaza52 is there a lein new x equivalent for tools.deps #2019-08-18 08:57:54 dominicm Clj-new #2019-08-18 10:26:20 murtaza52 thanks guys #2019-08-18 13:08:07 murtaza52 going through the docs, not able to understand the significance of the -A alias, when should it be used and when not ? Bcoz it seems it can do what all other aliases can do and vice versa, so either only it is needed or others are needed. #2019-08-18 13:26:42 alexmiller Generally you can just use -A #2019-08-18 13:27:00 alexmiller There are some weird cases where you might only parts of an alias to take effect #2019-08-18 14:37:40 murtaza52 thanks, that explains it #2019-08-20 10:33:55 unbalanced is there any way that you can specify extra deps on the command line without using an alias?
I've developed a template to help my coworkers quick-start on clojurescript but explaining to them how to add an alias for clj-new to their .clojure/deps.edn is just one more annoyance for them to figure out. I'd like to be able to do something like
clj -m clj-new.create --middleware "[seancorfield/clj-new]" some.example
#2019-08-20 17:43:19 souenzzo why not clj -Sdeps '{:deps,{seancorfield/clj-new,{:mvn/version,"0.5.5"}}}' -m clj-new.create some.example #2019-08-20 22:01:54 unbalanced that's it! #2019-08-20 10:34:51 unbalanced or perhaps read a deps.edn from stdin that I could have them just paste in via file descriptor? #2019-08-20 10:37:59 unbalanced clj -i -m clj-new.create some.example <(cat << eof
{:aliases :new {:extra-deps {seancorfield/clj-new
{:mvn/version "0.5.5"}}
:main-opts ["-m" "clj-new.create"]}
eof
)
something like that would also work but I haven't been able to figure that one out yet either #2019-08-20 10:43:31 vlaaad it's worth explaining IMHO: there is no magic, everything is simple and configurable #2019-08-20 11:28:37 alexmiller -Sdeps lets you add an extra deps.edn on the command line #2019-08-20 19:05:28 unbalanced does anyone have an example of the -Sdeps syntax? Is a string? #2019-08-20 19:07:21 lispyclouds clj -Sdeps "{:deps {clj-http {:mvn/version \"3.10.0\"}}}" ? #2019-08-20 19:07:33 unbalanced the command I'm hoping for is
clj -Sdeps '{seancorfield/clj-new {:mvn/version {"0.5.5"}}' -m clj-new.create project.name #2019-08-20 19:07:38 unbalanced ahhh ok #2019-08-20 19:07:50 unbalanced thanks I wasn't sure the exact syntax #2019-08-20 19:17:35 alexmiller it is literally the text of a deps.edn file #2019-08-20 21:44:05 unbalanced outstanding ... exactly what I needed, works exactly as advertised, thank you! #2019-08-20 22:32:22 cfleming I’m having a problem loading a namespace which requires t.d.a, using version 0.7.535. I’m getting the following error:
Syntax error compiling at (clojure/tools/deps/alpha/reader.clj:98:5).
No such var: io/read-edn
#2019-08-20 22:32:45 cfleming This seems weird, because io/read-edn does exist. #2019-08-20 22:33:04 cfleming I’m just running a standard REPL, classpath looks ok to me. #2019-08-20 22:35:49 cfleming ( "clojure/tools/deps/alpha/reader.clj")
=>
#object[java.net.URL
0x2bf95d38
"jar:file:/Users/colin/.gradle/caches/modules-2/files-2.1/org.clojure/tools.deps.alpha/0.7.535/11de58e45ac07ea473f2322f335ef6d162dffde9/tools.deps.alpha-0.7.535.jar!/clojure/tools/deps/alpha/reader.clj"]
( "clojure/tools/deps/alpha/util/io.clj")
=>
#object[java.net.URL
0x2b2beaaa
"jar:file:/Users/colin/.gradle/caches/modules-2/files-2.1/org.clojure/tools.deps.alpha/0.7.535/11de58e45ac07ea473f2322f335ef6d162dffde9/tools.deps.alpha-0.7.535.jar!/clojure/tools/deps/alpha/util/io.clj"]
#2019-08-20 22:39:07 hiredman maybe check (get (ns-publics ') 'read-edn) #2019-08-20 22:39:38 hiredman the namespace could have been created somehow and added to the lib-map, without loading the source file, stopping require from actually loading it #2019-08-20 22:39:39 cfleming (get (ns-publics ') 'read-edn)
=> nil
(ns-publics ')
=>
{printerrln #',
write-file #',
slurp-edn #'
#2019-08-20 22:40:29 cfleming All the vars are there except that read-edn #2019-08-20 22:42:04 hiredman no install-deps either (not sure if that is in 0.7.535, I am looking at master) #2019-08-20 22:42:45 cfleming No, I don’t see that in the 0.7.535 rev #2019-08-20 22:44:08 cfleming I can’t see anything in the source doing anything funky like unmapping vars. #2019-08-20 22:44:48 hiredman read the diff wrong, install-deps is a different namespace #2019-08-20 22:45:57 cfleming If I just restart the REPL and then require http://c.t.d.a.u.io directly, then the publics are wrong right off the bat. #2019-08-20 22:46:49 cfleming Is there a good way to diagnose where the ns is actually being loaded from? #2019-08-20 22:48:20 hiredman you could change the require function to print debugging information(a stacktrace, maybe clojure.lang.Compiler/LOADER, but I guess that is just a dynamic classloader) every time it is called #2019-08-20 22:51:13 hiredman you could also see what (.getResource (RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj") returns, maybe check for aot'ed classfiles too #2019-08-20 22:51:48 cfleming (.getResource (RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj")
=>
#object[java.net.URL
0x37339a
"jar:file:/Users/colin/.gradle/caches/modules-2/files-2.1/org.clojure/tools.deps.alpha/0.7.535/11de58e45ac07ea473f2322f335ef6d162dffde9/tools.deps.alpha-0.7.535.jar!/clojure/tools/deps/alpha/util/io.clj"]
#2019-08-20 22:51:59 cfleming The same as before, looks like it’s coming from the right jar. #2019-08-20 22:52:30 hiredman you could also look in the jar and see if has the right file contents 🙂 #2019-08-20 22:53:41 cfleming It does 🙂 #2019-08-20 22:58:38 hiredman something like (alter-var-root #'require (fn [orig]
(fn [& args]
(println (.getResource (clojure.lang.RT/baseLoader) "clojure/core.clj"))
(apply orig args))))
with the funky namespace's resource name instead of clojure/core.clj and then requiring the namespace might show something #2019-08-20 23:00:21 cfleming (alter-var-root #'require (fn [orig]
(fn [& args]
(println (.getResource (clojure.lang.RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj"))
(apply orig args))))
=> #object[user$eval1256$fn__1257$fn__1258 0x1a272414 " #2019-08-20 23:01:17 cfleming I’m not sure why require would be called twice there. #2019-08-20 23:03:06 hiredman that may be a clue #2019-08-20 23:03:26 cfleming And if I use:
(alter-var-root #'require (fn [orig]
(fn [& args]
(println (slurp (.getResource (clojure.lang.RT/baseLoader) "clojure/tools/deps/alpha/util/io.clj")))
(apply orig args))))
then the correct file contents are printed out. #2019-08-20 23:03:36 hiredman I think I've seen that behavior when the namespace in question has already been loaded (why that results in two require calls I have no idea) #2019-08-20 23:03:37 cfleming (twice) #2019-08-20 23:04:14 hiredman in a fresh repl before requiring anything maybe try the ns-publics call for the namespace again #2019-08-20 23:05:08 cfleming In a plain clj -Adeps REPL, where the publics are correct, it’s also printed twice. #2019-08-20 23:05:50 cfleming Yeah, (find-ns ') returns nil in a fresh REPL. #2019-08-20 23:09:17 cfleming Weirdly though, requiring something like clojure.zip only prints once. #2019-08-20 23:09:18 alexmiller -Adeps loads an uberjar #2019-08-20 23:09:32 alexmiller That won’t play nice with the lib jar #2019-08-20 23:09:58 cfleming No, I was just using -Adeps without a deps.edn to test in a clean REPL. #2019-08-20 23:10:10 cfleming i.e. to compare to what I was seeing in the problematic one. #2019-08-20 23:11:02 alexmiller Nothing unusual about how this ns is defined or loaded afaik #2019-08-20 23:11:20 cfleming No, I can’t see anything. #2019-08-20 23:13:44 hiredman could you have some aot classfiles somewhere on the classpath, if they have a newer last modified they will be loaded instead of the clojure source #2019-08-20 23:14:32 cfleming I’m wondering that. I have the debugger set up, and Compiler.load() is never called. #2019-08-20 23:14:32 alexmiller I don’t think that fn was added till 541 #2019-08-20 23:16:51 cfleming Ugh, that was the problem. #2019-08-20 23:17:00 cfleming Rogue AOT classes sitting around. #2019-08-20 23:17:06 cfleming Sorry for the noise. #2019-08-20 23:17:45 cfleming Thanks for the help! #2019-08-20 23:18:34 cfleming I’m not sure why there would have been AOT classes for the .io namespace but not the one calling it, though. I guess it must have been a previous transitive compilation. #2019-08-21 20:44:48 alexmiller clj 1.10.1.469 and tools.deps.alpha 0.7.541 released... #2019-08-21 20:44:57 alexmiller - FIX: exclusions were not canonicalized and could fail to match and exclude
- PERF: cache Maven session and use Maven session cache
- FIX: Remove slf4j-nop as dependency of tools.deps.alpha
#2019-08-21 20:48:16 alexmiller you may also need to add slf4j-nop as a dep if you were previously relying on getting it via tools.deps.alpha #2019-08-21 23:36:15 seancorfield @alexmiller Under what circumstances might I expect with-session to make a difference? #2019-08-21 23:37:50 seancorfield I just updated clj-new to use the latest t.d.a and wrapped the pair of t.d.a calls in that, but saw pretty much no difference in time spent generating a template (from an empty M2 repository each time). (session/with-session
(-> (deps/resolve-deps deps resolve-args)
(deps/make-classpath (:paths deps) {})
(str/split (re-pattern java.io.File/pathSeparator))
(->> (run! pom/add-classpath)))))
#2019-08-21 23:49:57 alexmiller It will only help if you’re talking to repos of course #2019-08-21 23:50:15 alexmiller But in particular it makes a difference taking to s3 as it’s slow #2019-08-21 23:51:12 alexmiller For building a large classpath (without cp cache), when talking to s3 it was as much as 50% better #2019-08-21 23:51:29 alexmiller And in other cases, no observable difference #2019-08-21 23:54:55 seancorfield Ah, OK. So it's harmless at worst, and might be helpful for some small class of clj-new users...? #2019-08-22 03:21:46 alexmiller yes, should be no worse, and occasionally much better #2019-08-22 21:46:48 dominicm Afk, but default deps come to mind #2019-08-22 21:54:55 danielcompton Thanks! :override-deps also look promising #2019-08-22 21:57:00 dominicm That sounds right too #2019-08-22 21:57:16 dominicm I think the downside is the alias requirement #2019-08-22 21:59:53 danielcompton Yeah, was just going to say that #2019-08-22 22:00:11 danielcompton Doesn't communicate it in the same way that managed dependencies does #2019-08-22 22:02:43 alexmiller default-deps is addressing the same use case as dependency management but there is still work to do to make it actually useful #2019-08-22 22:29:05 danielcompton Can you use it at the root of a deps.edn or does it require an alias? #2019-08-22 23:05:11 alexmiller Alias #2019-08-22 23:42:07 seancorfield @danielcompton I added an answer explaining how we use :override-deps at World Singles Networks -- because we use it extensively! #2019-08-22 23:43:01 danielcompton Thanks to both of you! This feels like a great use-case for Ask Clojure to me #2019-08-23 16:40:29 aaelony I shared this link at work and the feedback was that it is a PHP site... #2019-08-23 16:50:02 seancorfield Alex has addressed that already. It is an off-the-shelf solution that is easy to maintain/administer. #2019-08-23 16:50:43 seancorfield And provides all the functionality needed for the task. There is no Clojure/Script equivalent to use (but folks are welcome to write such things and offer them up to Cognitect/the public). #2019-08-23 17:07:09 aaelony makes sense to me #2019-08-23 14:53:45 astrashe Is there an example anywhere that shows how to use deps.edn with a local java library? I need a local java library to deal with a protobuf #2019-08-23 14:59:07 andy.fingerhut If the Java library is compiled to a JAR file, then you should be able to add it as a local dependency using something like this: {:deps {somejavalib {:local/root "/path/to/the/jar/file.jar"}}} If you search for ":local" on the reference doc page for deps, you should find that, too: https://clojure.org/reference/deps_and_cli #2019-08-23 15:00:43 astrashe @andy.fingerhut Thanks! The JAR file seems like the way to go #2019-08-23 15:03:25 vlaaad I wish future clojure releases focused more on java interop, particularly around proxies and class creation. tools-deps is a great value proposition, with the downside being that using .java files in the same project is clumsy. I really don't want to go back to lein... #2019-08-23 15:21:23 alexmiller I think that's unlikely given that Clojure is in philosophical opposition to concrete derivation #2019-08-23 15:33:48 vlaaad I think that's what I need actually, having troubles with proxy not having access to some protected stuff and using reflection without type hinting this.. #2019-08-24 07:44:55 dharrigan Is there any chance that "add-libs" branch on tools deps can be rebased from master so that the latest changes on master for the just released version can be brought in? The "add-lib" functionality is really really useful to have (along with the removal of the slf4j-nop library....) #2019-08-24 07:44:55 dharrigan Is there any chance that "add-libs" branch on tools deps can be rebased from master so that the latest changes on master for the just released version can be brought in? The "add-lib" functionality is really really useful to have (along with the removal of the slf4j-nop library....) #2019-08-24 17:03:34 seancorfield @U064X3EF3 is usually pretty good about merging master back to the add-lib branch -- esp. once we add-libbers ask for it 🙂 #2019-08-24 21:18:33 alexmiller I merged it #2019-08-24 21:18:40 alexmiller merged master to it that is #2019-08-25 00:09:44 seancorfield You rock! Thank you! #2019-08-25 00:10:47 seancorfield @U11EL3P9U @U1EQNSHL4 SHA 148cab8f154711f8495bd632ebdc53a14dff7f09 to save you both going to look it up. I'm updating my dot-clojure repo right now... #2019-08-25 06:44:46 dharrigan w00t w00t! #2019-08-25 10:40:30 just.sultanov thank you @U04V70XH6 @U064X3EF3 #2019-08-25 10:53:38 just.sultanov In my current project, I need to add libraries to the classpath not only in the REPL at development time. Therefore, I made small changes to the function add-loader-url. #2019-08-25 10:56:33 just.sultanov (defn- add-loader-url
"Add URL string or URL to the highest level DynamicClassLoader or URLClassLoader URL set."
[url]
(let [url (if (string? url) (URL. url) url)
thread (Thread/currentThread)
cl (loop [loader (.getContextClassLoader thread)]
(let [parent (.getParent loader)]
(if (instance? DynamicClassLoader parent)
(recur parent)
loader)))]
(if (instance? DynamicClassLoader cl)
(.addURL ^DynamicClassLoader cl url)
(let [loader (.getContextClassLoader thread)
urls (into-array URL [url])
cl (URLClassLoader. urls loader)]
(.setContextClassLoader thread cl))))) #2019-08-24 07:44:57 dharrigan 🙂 #2019-08-24 07:45:18 dharrigan Thank you 🙂 #2019-08-24 21:04:46 seancorfield seancorfield/clj-new 0.7.8 -- minor maintenance release: fixes -v / --verbose option (betcha didn't know it was broke, eh?), updates dependencies and pins some new dependencies to avoid weird conflicts in some (Leiningen) templates -- I think that's due to the difference between tools.deps and lein/`boot` in terms of resolving conflicts (gotta love those pesky Jackson libraries!). #2019-08-26 18:34:40 seancorfield seancorfield/depstar 0.3.2 -- encountering an unknown file type in a dependency no longer prevents building a JAR (you'll either get a warning or not, depending on whether the name of the file is on the "excluded" list -- you can see more of what is going on either via the -v option or by enabling debugging). #2019-08-27 16:04:41 nathanmarz trying to use a git coordinate on a repo that has .clj and .java source files, but trying to use a namespace from the dependency in a repl fails because the java source isn't being compiled #2019-08-27 16:04:51 nathanmarz is there a way to handle this? #2019-08-27 16:10:26 alexmiller With deps, no #2019-08-27 16:10:53 alexmiller For projects that require a build step, you’ll need to use an artifact #2019-08-27 16:10:58 hiredman tools.deps doesn't provide any built in way to run a build step (like calling javac), I think when it has come up in the past people have talked about calling out to javac, but I am not sure if anyone has wrapped that up into something re-usable #2019-08-27 16:13:28 hiredman and I wouldn't hold my breath for something that solves that in the general case, because you would really need something that knows how to build arbitrary java projects, much easier to rely on built artifacts #2019-08-28 18:28:53 onetom I'm wondering how can I test running clj for the 1st time in a deps.edn project, without rm -r ~/.m2
I remember in the past I could just do something like env M2_HOME=$(realpath .)/m2 boot repl or something like that, but now this is the only way I found to affect that path:
mvn -Dmaven.repo.local=$(realpath .)/m2 help:evaluate -Dexpression=settings.localRepository
but if I try something similar with clj, it still seem to use the default location (since it's not downloading anything):
rm -r .cpcache; time clj -J-Dmaven.repo.local=$(realpath .)/m2 -e 1
#2019-08-28 18:37:31 onetom And, btw, I'm still seeing abysmal performance when accessing Maven Central from Hong Kong.
After rm -r ~/.m2, it took 9minutes to download 25MB of jars (just clojure, netty, aleph, compojure).
While my connection is shit and hardly ever goes over 2MByte/s and it's below 1MB/s mostly, 9minutes is unreasonable.
It seems to me that the main reason is the number of requests, which are also made serially (and might not even use keep-alive?)
So I don't think only S3 is slow. #2019-08-28 18:40:45 onetom I've also checked the related DNS chain:
. 1735 IN CNAME .
. 21599 IN CNAME .
. 29 IN A 151.101.196.215
and that IP points to San Jose. #2019-08-28 18:41:54 onetom I had similar issues with the nixos binary caches. When they switched from AWS CloudFront to Fastly (which is significantly cheaper, plus they actually sponsor them), then the performance from Hong Kong dropped drastically. #2019-08-28 18:42:01 alexmiller you can set :mvn/local-repo in your deps.edn #2019-08-28 18:42:31 alexmiller to use a different (presumably empty) local repo dir #2019-08-28 18:42:33 onetom so not from the command line or environment? #2019-08-28 18:42:48 alexmiller you can clj -Sdeps '{:mvn/local-repo "foo"}' #2019-08-28 18:42:52 alexmiller so that's command line #2019-08-28 18:43:09 onetom but im not talking about specifying dependencies #2019-08-28 18:43:19 onetom im talking about specifying the maven cache directory #2019-08-28 18:43:32 alexmiller the local repo is the maven cache directory #2019-08-28 18:43:55 alexmiller usually it's ~/.m2/repository #2019-08-28 18:44:04 onetom indeed, it works #2019-08-28 18:44:29 alexmiller are you using latest clj? #2019-08-28 18:44:42 alexmiller 1.10.1.469 #2019-08-28 18:44:50 onetom where is this documented? #2019-08-28 18:44:59 alexmiller clj -Sverbose will tell you #2019-08-28 18:46:36 alexmiller we are making use of maven session caches as of that version, so would be interested in whether that makes a difference for you #2019-08-28 18:46:36 onetom version = 1.10.1.466 #2019-08-28 18:46:58 alexmiller it should mostly cut down on repeated download of metadata files though, which are pretty tiny #2019-08-28 18:47:07 onetom it would make sense to use a more common option for this, like -v; i keep forgetting it 😕 #2019-08-28 18:47:46 onetom thanks, i will give it a go #2019-08-28 18:47:59 alexmiller jars are serially downloaded #2019-08-28 18:49:21 alexmiller there are ways to change that, but, it's somewhat involved #2019-08-28 18:50:34 onetom is there an off the shelf solution for setting up a maven repo manager specifically for clojure usage?
the https://maven.apache.org/repository-management.html page mentions a lot of options.
a few years ago i've tried a few of them but i was struggling to make them work.
it should be practically a caching http proxy...
i was really hoping to find some small, turn-key solution, which i would run in the cloud, so i can share it between the office an home #2019-08-28 18:51:49 alexmiller there are several turn-key products for this, not sure if any qualify as "small" #2019-08-28 18:52:13 alexmiller I've used Nexus a lot #2019-08-28 18:52:52 alexmiller but there is also JFrog and Artifactory #2019-08-28 18:53:09 alexmiller sorry, those are the same #2019-08-28 18:53:15 alexmiller Artifactory is by JFrog #2019-08-28 18:53:17 hiredman you can just use a caching http proxy too, you just don't get the nice management interfaces, and the ability to merge sources, etc #2019-08-28 18:54:09 onetom latest nixpkgs contains 1.10.1.469 on its master branch. downloading it now:
uhu:multiboxx onetom$ nix-shell -I nixpkgs=~/nixpkgs/ -p clojure
these paths will be fetched (307.02 MiB download, 990.58 MiB unpacked):
/nix/store/0d8mpzq2dah05xqd6i1c9g02blvsvcnj-bash-interactive-4.4-p23-man
looks like it will take about 10minutes.
i will try a clean download of the same 25MB dependencies afterwards #2019-08-28 18:54:11 alexmiller there are some benefits to using a maven-aware product I think #2019-08-28 18:55:20 alexmiller I guess one question I have is why you care what the perf is if you do it just once and then cache it forever? #2019-08-28 18:55:29 onetom @hiredman and what would you recommend as a caching http proxy? would it work with https? so for example datomic can be cached too? #2019-08-28 18:56:57 onetom @alexmiller these things never happen only once.
i have multiple personal computers, multiple office computers, more and more servers and hoping to add more colleagues too.
these caches should be primed on all those filesystems and having a bad 1st experience doesn't help... #2019-08-28 18:56:58 hiredman I think it could be done, but it is a trade off, if you really want something a lean caching solution a caching proxy would be that, but you will have to invest time getting it working #2019-08-28 18:57:20 onetom also on CI systems it's good to be able to build stuff afresh from time to time at least #2019-08-28 18:57:26 alexmiller I'm not trying to be flip about it, obviously perf matters, just trying to probe a little closer #2019-08-28 18:58:00 alexmiller the structure of maven is designed that you don't need to refresh - these are immutable, uniquely versioned artifacts #2019-08-28 18:58:16 alexmiller CI is one use case where we see this come up #2019-08-28 18:58:20 onetom more specifically, i have 2 clojure projects im working on at the moment and i just wanted to know how much are their dependencies #2019-08-28 18:58:21 hiredman at my last job we had a nexus setup that aggregated a number of different repos, and our builds were setup to only check our nexus, and that worked pretty well #2019-08-28 18:58:43 alexmiller but most CI allow you to keep a maven cache alive #2019-08-28 18:59:22 onetom i could have built an uber jar, but my knowledge is outdated on how to do that with t.d
im just getting back to using clojure after a ~2yr break (when i was working with ethereum and js...) #2019-08-28 18:59:57 alexmiller there might also be better maven central mirrors you could use from HK, not sure #2019-08-28 19:00:22 alexmiller clj (latest) supports Maven mirrors #2019-08-28 19:01:38 onetom i looked into the mirrors but most of them doesn't work.
there is a UK one which is not offline but to do a performance test, i wanted to know how can i relocate the m2 cache, since it's not fun to delete my ~/.m2 directory 🙂 #2019-08-28 19:01:47 alexmiller there's one on google storage I know #2019-08-28 19:02:29 alexmiller <settings>
<mirrors>
<mirror>
<id>google-maven-central</id>
<name>Google Maven Central</name>
<url>https://maven-central.storage.googleapis.com</url>
<mirrorOf>central</mirrorOf>
</mirror>
</mirrors>
</settings> #2019-08-28 19:02:56 onetom never seen that being mentioned anywhere #2019-08-28 19:03:58 alexmiller looks like they have an asia-pacific one there too #2019-08-28 19:07:15 onetom thanks a lot, @alexmiller !
if these work, it will really help with years of frustration 🙂 #2019-08-28 19:08:05 alexmiller please report back, would be very interested to hear if those are any better #2019-08-28 19:12:14 onetom hmm... it took one minute to download 18MB, but only the 1st dependency was printed
[nix-shell:/Volumes/Data/lab/multiboxx]$ clj -Sverbose
version = 1.10.1.469
install_dir = /nix/store/0k1f3llrx4aggbgx7rhh70mrardqvrgz-clojure-1.10.1.469-prefix
config_dir = /Users/onetom/.clojure
config_paths = /nix/store/0k1f3llrx4aggbgx7rhh70mrardqvrgz-clojure-1.10.1.469-prefix/deps.edn /Users/onetom/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1871601414.cp
Refreshing classpath
[nix-shell:/Volumes/Data/lab/multiboxx]$ time clj -e 1
Downloading: riddley/riddley/0.1.12/riddley-0.1.12.pom from
1
real 1m18.496s
user 0m13.404s
sys 0m0.742s
[nix-shell:/Volumes/Data/lab/multiboxx]$ ls m2/
cheshire/ clj-tuple/ commons-io/ me/ riddley/ tigris/
clj-http/ com/ commons-logging/ org/ seancorfield/
clj-soup/ commons-codec/ etaoin/ potemkin/ slingshot/
#2019-08-28 19:13:56 onetom ah, nvm, i forgot to rm -r m2 .cpcache beforehand #2019-08-28 19:17:53 onetom 2nd run is printing all the downloaded deps:
[nix-shell:/Volumes/Data/lab/multiboxx]$ rm -r m2 .cpcache/; time clj -e 1; du -hsc m2
Downloading: org/clojure/clojure/1.10.1/clojure-1.10.1.pom from
...
Downloading: org/apache/httpcomponents/httpmime/4.5.2/httpmime-4.5.2.jar from
Downloading: com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.7.5/jackson-dataformat-smile-2.7.5.jar from
Downloading: org/clojure/data.codec/0.1.0/data.codec-0.1.0.jar from
1
real 1m30.756s
user 0m16.289s
sys 0m1.102s
18M m2
18M total
this is magnitudes better!
i guess i should make a counter test with the old version just to confirm im not just experiencing some change in network conditions #2019-08-28 19:18:59 alexmiller using -Sforce will let you force a classpath recompute (don't need to nuke your .cpcache then) #2019-08-28 19:19:31 alexmiller is that difference with just the new clj version or the new clj version + mirror? #2019-08-28 19:21:08 onetom only the clj version is different #2019-08-28 19:21:30 onetom this is my deps.edn:
{:paths ["src" "rsc"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/data.csv {:mvn/version "0.1.4"}
clj-soup/clojure-soup {:mvn/version "0.1.3"}
me.raynes/fs {:mvn/version "1.4.6"}
org.clojure/java.jdbc {:mvn/version "0.7.9"}
org.xerial/sqlite-jdbc {:mvn/version "3.28.0"}
com.microsoft.sqlserver/mssql-jdbc {:mvn/version "7.2.2.jre8"}
seancorfield/next.jdbc {:mvn/version "1.0.5"}
etaoin {:mvn/version "0.3.5"}}
:aliases {:test {:extra-paths ["test"]}}
:mvn/local-repo "m2"
:mvn/repos {
"central" {:url ""}
;"uk" {:url ""}
"clojars" {:url ""}
}}
#2019-08-28 19:22:20 onetom next thing after the old version finishes is to try the google mirror #2019-08-28 19:23:04 onetom which i guess, i can simply do by replacing the :url for the "central" entry to the google mirror's url, right? #2019-08-28 19:23:44 onetom but currently im seeing 7-30kbyte/s download rates with the old version, so it will take awhile... #2019-08-28 19:24:41 onetom nope, it took only ~3mins:
uhu:multiboxx onetom$ clojure -Sverbose
version = 1.10.1.466
install_dir = /nix/store/m2v1xd7cybi544jyl77w3mzjw1xklw41-clojure-1.10.1.466-prefix
config_dir = /Users/onetom/.clojure
config_paths = /nix/store/m2v1xd7cybi544jyl77w3mzjw1xklw41-clojure-1.10.1.466-prefix/deps.edn /Users/onetom/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1871601414.cp
Refreshing classpath
^Cuhu:multiboxx onetom$ rm -r m2 .cpcache/; time clj -e 1; du -hsc m2
rm: cannot remove '.cpcache/': No such file or directory
...
Downloading: org/clojure/data.codec/0.1.0/data.codec-0.1.0.jar from
1
real 3m21.764s
user 0m18.844s
sys 0m1.207s
18M m2
18M total
#2019-08-28 19:25:52 onetom still, it's quite significant difference, so thanks a lot for telling me about this enhancement and for implementing these improvements! #2019-08-28 19:32:51 onetom 2/3 of the dependencies come from maven central and the rest is clojars:
uhu:multiboxx onetom$ rg -c maven-central-asia 466-deps.txt
71
uhu:multiboxx onetom$ rg -c 466-deps.txt
22
uhu:multiboxx onetom$ wc -l 466-deps.txt
92 466-deps.txt
(it almost adds up :) #2019-08-28 19:35:55 onetom that consistently seems to be a more than 2x difference! #2019-08-28 19:36:15 alexmiller pretty big combined change! :) #2019-08-28 19:36:18 onetom although it's possible that im warming up some CDN caches... #2019-08-28 19:37:04 onetom do you happen to know any asia-pacific clojars mirror by any chance? 🙂 #2019-08-28 19:37:24 onetom i will try this mirror on my other project too #2019-08-28 19:39:35 alexmiller I don't believe there is anything region specific. they do have a cdn mirror though #2019-08-28 19:39:54 alexmiller I have no idea of difference in perf though #2019-08-28 19:48:45 onetom this google mirror understands HTTP2:
uhu:reap onetom$ curl -vso /dev/null --http2 2>&1 | rg -i http2
* Using HTTP2, server supports multi-use
is the HTTP client in clj talking HTTP2? #2019-08-28 19:49:22 alexmiller that's several layers below the code I'm in, so don't know #2019-08-28 19:49:30 alexmiller it would not surprise me if the answer was no #2019-08-28 19:53:34 alexmiller the maven resolver uses org.apache.httpcomponents/httpclient and the version it's pinned on does not look like it supports http 2 (the newest series does though) #2019-08-28 19:54:02 alexmiller so my strong guess would be: no, but it potentially could #2019-08-28 20:06:30 onetom clj 1.10.1.466
mirror: https://maven-central-asia.storage-download.googleapis.com/repos/central/data/
[nix-shell:~/lab/reap]$ rm -r m2; time clj -Sforce -e 1; du -hsc m2
...
real 3m33.490s
user 0m26.609s
sys 0m1.819s
24M m2
24M total
that's less of a difference, but still quite noticable (~20%)
anyway, enough testing for tonight; it's 4am in HK, so i shall sleep.
thanks again for your attention and advice! #2019-08-29 04:33:28 onetom it happened again.
when i've added the :mvn/local-repo "m2" to my deps.edn and ran time clj -e 1 for the 1st time, i've only seen some of the dependencies being printed, while it was actually downloading all of them:
Downloading: ring/ring-jetty-adapter/1.6.3/ring-jetty-adapter-1.6.3.pom from
Downloading: crypto-equality/crypto-equality/1.0.0/crypto-equality-1.0.0.jar from
Downloading: potemkin/potemkin/0.4.5/potemkin-0.4.5.jar from
1
real 1m42.094s
user 0m16.879s
sys 0m0.933s
raven:reap onetom$ du -hsc m2
22M m2
22M total
#2019-08-29 04:37:16 onetom after an rm -r m2, on subsequent run, i could see all of the deps printed as they were being downloaded and the size of the maven cache ended up being the same:
raven:reap onetom$ rm -r m2
raven:reap onetom$ time clj -Sforce -e 1
Downloading: org/clojure/data.json/0.2.6/data.json-0.2.6.pom from
...
Downloading: com/google/code/gson/gson/2.7/gson-2.7.jar from
1
real 0m38.062s
user 0m16.855s
sys 0m1.167s
raven:reap onetom$ du -hsc m2
22M m2
22M total
something, somewhere might be caching these artefacts, because the 2nd time was so much faster...
but MAYBE the first time is slower because of the same bug, which causes only a few of the downloaded dependencies being printed! #2019-08-29 04:38:29 onetom for the record, my deps.edn looks like this:
{:paths ["src" "rsc" "java" "classes"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
clj-http {:mvn/version "3.10.0"}
cheshire {:mvn/version "5.1.1"}
org.clojure/data.json {:mvn/version "0.2.6"}
com.squareup.wire/wire-runtime {:mvn/version "3.0.0-rc01"}
com.google.protobuf/protobuf-java {:mvn/version "3.9.1"}
com.google.protobuf/protobuf-java-util {:mvn/version "3.9.1"}
zprint {:mvn/version "0.4.16"}
ring/ring-core {:mvn/version "1.7.1"}
ring/ring-devel {:mvn/version "1.7.1"}
ring/ring-jetty-adapter {:mvn/version "1.6.3"}
aleph {:mvn/version "0.4.6"}
compojure {:mvn/version "1.6.1"}
hiccup {:mvn/version "1.0.5"}
;org.clojure/clojurescript {:mvn/version "1.10.520"}
}
:aliases {:test {:extra-paths ["test"]
:extra-deps {
;clj-http-fake {:mvn/version "1.0.3"}
}}}
:mvn/local-repo "m2"
;
:mvn/repos {"central" {:url
""
#_""}
"clojars" {:url ""}}}
#2019-08-29 04:39:08 onetom and my clj version info looks like this:
raven:reap onetom$ time clj -Sverbose -e 1
version = 1.10.1.469
install_dir = /nix/store/bqs5l2mc2jlj5z1a21ix0sz0ak8cvn0l-clojure-1.10.1.469-prefix
config_dir = /Users/onetom/.clojure
config_paths = /nix/store/bqs5l2mc2jlj5z1a21ix0sz0ak8cvn0l-clojure-1.10.1.469-prefix/deps.edn /Users/onetom/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1871601414.cp
1
real 0m0.858s
user 0m2.045s
sys 0m0.131s
#2019-08-29 04:39:46 onetom /Users/onetom/.clojure/deps.edn is all commented out #2019-08-29 13:35:40 hoynk Is it possible to just download the dependencies of my project, without running anything? Like with lein deps. #2019-08-29 13:36:25 dominicm Doing -Spath will print the classpath, and won't execute anything. #2019-08-29 13:37:09 hoynk Nice trick, thanks! #2019-08-30 21:27:34 andy.fingerhut It basically uses a bunch of aliases, some prefixed with ":clj-" and others with ":cljs-", for similar functionality in each Clojure flavor. The best alternative I could imagine to that approach would be to have two different directories with different deps.edn files, one for Clojure/Java, the other for ClojureScript. #2019-08-30 21:29:14 andy.fingerhut Certainly open to other ideas if anyone thinks of improvements to that approach. #2019-08-30 21:33:16 Felipe Cortez should add-lib work for every lib or are there limitations?
(require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
(add-lib 'cheshire {:mvn/version "5.9.0"})
(require '[cheshire.core :as json])
fails with
Syntax error (ClassNotFoundException) compiling new at (cheshire/factory.clj:57:11).
com.fasterxml.jackson.core.async.ByteArrayFeeder
Syntax error compiling at (cheshire/core.clj:1:1).
namespace 'cheshire.factory' not found
#2019-08-30 21:36:19 dominicm Does it work if you include it in your deps.edn? #2019-08-30 21:36:30 Felipe Cortez yep #2019-08-30 21:47:00 seancorfield I ran into something like this with clj-new and the Luminus template... as I recall it's a peculiarity of the Jackson library as far as its dependencies go... #2019-08-30 21:47:27 seancorfield (or maybe it's a bug in how Cheshire is packaged?) #2019-08-30 21:47:55 seancorfield I ended up needing to explicitly depend on various Jackson libraries in order to get that to succeed... #2019-08-30 21:50:17 alexmiller add-lib will interpret new libraries in the context of your existing libs #2019-08-30 21:50:44 alexmiller so if you already have an existing version of jackson on the classpath, and you add cheshire, it (can't) override the versions you already have #2019-08-30 21:51:00 alexmiller if cheshire needs a newer version than the one you have, it may not work #2019-08-30 21:51:35 alexmiller whereas if you had run a deps resolution on the full set of deps, it might have been able to resolve to a better set of deps #2019-08-30 21:52:55 alexmiller jackson tends to be involved in 75% of dependency conflicts, so it's always a likely culprit #2019-08-30 21:52:57 seancorfield Yeah, that sounds like the problem -- Cheshire/Jackson version mismatches. #2019-08-30 21:54:08 alexmiller this is a totally normal thing that can happen with add-libs #2019-08-30 21:54:12 seancorfield S3 wagon lib pulls in Jackson 2.5.5 #2019-08-30 21:56:37 Felipe Cortez makes sense, thanks! #2019-08-30 21:57:29 seancorfield @clojurians884 Here's a combination that works: (! 508)-> clj -A:deps -Sdeps '{:deps {com.fasterxml.jackson.core/jackson-core {:mvn/version "2.7.5"}}}'
Clojure 1.10.1
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'cheshire {:mvn/version "5.6.3"})
true
user=> (require 'cheshire.core)
nil
user=>
#2019-08-30 21:57:59 seancorfield Note the Jackson override on the command line. Without that, it won't work. #2019-08-30 21:58:23 Felipe Cortez awesome! thanks for the help 🙂 #2019-08-30 21:58:50 seancorfield You don't need to add-lib on the smile sub-lib either (edited to remove that) #2019-08-30 21:59:45 seancorfield If you want to use the latest Cheshire, you need Jackson 2.9.9: (! 510)-> clj -A:deps -Sdeps '{:deps {com.fasterxml.jackson.core/jackson-core {:mvn/version "2.9.9"}}}'
Clojure 1.10.1
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'cheshire {:mvn/version "RELEASE"})
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
true
user=> (require 'cheshire.core)
nil
user=>
#2019-08-30 22:00:19 seancorfield (I would love to excise Cheshire and Jackson from our code base for exactly this reason!) #2019-08-30 22:01:07 seancorfield (and the problem is because tools.deps brings in s3-wagon-private which in turn brings in Jackson 2.5.5 so you need an explicit override to make it all work) #2019-08-30 22:01:28 Felipe Cortez do you recommend using data.json instead? #2019-08-30 22:04:55 alexmiller I've got a spike of some work to use the cognitect s3 lib completely in place of the s3 wagon for tools.deps, would love to shift things over, but needs some work still (and prob streaming support in the s3 lib) #2019-08-30 22:05:29 alexmiller I think the cognitect stuff uses data.json, which is definitely not as fast as cheshire/jackson (but way less problematic) #2019-08-30 22:07:00 seancorfield If data.json does what you need -- and does it fast enough -- then, yes, use it instead of Cheshire to avoid the pit of dependency hell that is Jackson 🙂 But, in reality, you'll find it hard to avoid Cheshire as other libraries bring it in too... #2019-08-30 22:10:41 seancorfield (having said that, I'm now having a hard time finding a single library that depends on Cheshire! oh how I wish that http://cross-clj.info was still alive!) #2019-08-30 22:11:34 seancorfield (it was without - I think but that brings up an online casino that is squatting on the domain!) #2019-08-30 22:13:47 Felipe Cortez again, thanks for the help 😄 #2019-08-31 03:59:18 sogaiu is the purpose of -Dclojure.libfile=... in the invocation at the end of the clojure bash (powershell) script diagnostic / for debugging? #2019-08-31 04:36:37 seancorfield I'm not seeing anything in the code at first glance that uses it so... I don't know... but it's a good question to ask Alex when he's around (Tuesday, probably, given that it's Labor Day weekend here) @sogaiu #2019-08-31 04:40:54 sogaiu @seancorfield tnx! #2019-08-31 10:53:41 alexmiller It’s to communicate for add-lib or tools like it #2019-08-31 11:20:59 sogaiu @alexmiller thanks for the explanation -- i presume it's not considered an api one can rely on (i.e. the ability to access the info via the clojure.libfile system property), is that right? #2019-08-31 11:25:30 alexmiller It has to be considered optional. The impl in add-lib knows how to use several means to detect the lib set, including classpath introspection #2019-08-31 11:26:20 alexmiller That whole thing is potentially something we may add to Clojure itself, tbd (which is why add-lib is not final) #2019-08-31 11:35:59 cfleming I’d really like to use Deps as a lightweight way to manage deps for projects which aren’t always Clojure, or all Clojure. Currently there’s no way to globally exclude Clojure itself from the dependencies, correct? #2019-08-31 11:36:45 cfleming As I understand it, in current Deps versions the system deps.edn is always read from the bundled copy in t.d.a, and there’s also no way to override that - is that correct? #2019-08-31 11:38:11 cfleming I did try this in my deps.edn, but it didn’t work: {org.clojure/clojure {:mvn/version "1.10.1" :exclusions [org.clojure/clojure]}} #2019-08-31 11:40:03 cfleming Would there be interest in a patch adding global exclusions support to deps, say using a top-level :exclusions key? #2019-08-31 11:44:31 cfleming I also tried: {org.clojure/clojure {:local/root "empty"}} where empty/deps.edn is just {}, also no dice. #2019-08-31 14:50:35 kenny Global exclusions would be very useful for us too. #2019-08-31 15:29:20 alexmiller there is currently no way to "remove" a dependency from the top level, and clojure is included as a default dependency, so the practical effect is that clojure is always included #2019-08-31 15:29:53 alexmiller I don't consider this to be a major problem as the in-scope use case here is running Clojure programs (I don't consider clj to be a tool for generic deps mgmt) #2019-08-31 15:31:47 alexmiller that said, we do have a ticket for global exclusions and there certainly are times when that would be useful. similarly, it's pretty hard to use a different dep type (like local or git) to override the clojure version (due to issues in overrides of different dep type and version comparison), and that's something that I sometimes need to do #2019-08-31 15:33:32 alexmiller anyhow, I'm still thinking about the best solution to these #2019-08-31 20:46:26 cfleming Another use case I have is running a Clojure fork. #2019-08-31 21:08:24 alexmiller Yeah, that’s unintentionally difficult as I discovered recently when I was trying to do something similar #2019-09-01 19:37:43 andy.fingerhut If I run clj -Sdep '{:deps {org.clojure/clojure {:git/url "" :sha "...."}}}' I get Error: Could not find or load main class clojure.main. Is there a way to run arbitrary git commits of Clojure using clj command? #2019-09-01 19:38:05 andy.fingerhut I had an actual SHA hex string instead of ..... there, of cousre. #2019-09-01 19:43:07 andy.fingerhut This is with version 1.10.1.469 of clj on Linux #2019-09-01 19:44:40 andy.fingerhut Wait, I guess that cannot work, since it needs to compile Java code. Never mind. #2019-09-02 15:48:16 drewverlee whats a short summery of what the .cpcache folder contains? #2019-09-02 16:28:39 alexmiller .cp files (final classpaths) and .lib files (intermediate dep files) #2019-09-02 16:29:41 alexmiller clj either uses the cached cp file or has to spin a separate jvm with tools.deps to compute the classpath #2019-09-02 16:30:41 alexmiller It is truly a cache (key is a hash of a bunch of stuff) so it’s safe to rm it #2019-09-02 16:49:52 drewverlee makes sense, thanks for the explanation Alex. #2019-09-03 15:46:31 kenny Has anyone tried compiling the tools-deps with graal to increase performance? #2019-09-03 15:48:29 alexmiller there's really two stages in clj and the first one (cp computation) should be pretty amenable to it #2019-09-03 15:48:40 alexmiller it makes the install story (which is already complicated) much more complicated #2019-09-03 15:49:02 alexmiller and in most cases the result of cp computation is a) IO-bound and b) cached and skipped #2019-09-03 15:49:26 alexmiller given all that, exploring this is not a high priority for me #2019-09-03 16:05:33 kenny Primarily interested in the first one #2019-09-03 17:44:57 WhittlesJr Hi! I think I asked a while back about using self-signed certs for accessing a self-hosted maven repo over https with tools.deps. (The Leiningen equivalent is adding the path to the cert in :certificates [].) IIRC, at the time that I asked, this was not supported with tools.deps. I can't seem to find anything about this by googling or looking on JIRA. Could anyone give me an update on whether this works these days, or point me in the right direction for how to handle this scenario?
Relevant config in deps.edn:
:mvn/repos {"central" {:url "https://<my-nexus-server>/repository/maven-public/"}}
Everything works fine over http with tools.deps, and even with https with lein. But with https on tools.deps I get "unable to find valid certification path to requested target." #2019-09-03 18:07:43 alexmiller Nothing has been added in this area #2019-09-03 18:08:22 alexmiller Could add it to Java’s certs? #2019-09-03 18:17:31 WhittlesJr That's a good solution on most OSs, but I'm on NixOS which aims to be "immutable" so the keystore is read-only. I should probably look into adding the ability to add certs to the java keystore declaratively in NixOS.
Anyway, thanks for the update! #2019-09-03 18:00:53 WhittlesJr @sfyire I'm pretty sure that implies that the repo in question does not have a deps.edn (which it doesn't). To my knowledge, if you want to use :git/url, the target must have a deps.edn. #2019-09-03 18:01:54 Adrian Smith @alex.joseph.whitt ah thank you #2019-09-03 18:03:01 alexmiller You can tell it to do that #2019-09-03 18:05:37 Adrian Smith @alexmiller how would I do that? #2019-09-03 18:05:46 alexmiller If you add :deps/manifest :deps to the coord that overrides the autodetection (although you won’t get the deps in project.clj) #2019-09-03 18:06:20 alexmiller You could add those to the caller though if you don’t have access to modify the project #2019-09-03 18:06:43 alexmiller If you’ve forked it, then I’d just add a deps.edn #2019-09-03 18:06:58 Adrian Smith yeah, that's cool to know though #2019-09-03 18:07:05 WhittlesJr TIL #2019-09-03 18:17:51 seancorfield (I added the above as an answer on the ask site) #2019-09-03 18:25:41 Adrian Smith thank you #2019-09-05 22:30:03 Quest What's the equivalent of lein install for a clj deps.edn library? #2019-09-05 22:37:25 andy.fingerhut If you mean creating a JAR file of Clojure source code, I believe depstar is a good choice. #2019-09-05 22:50:56 Quest Appreciate the recommendation, easy enough to get a jar generated with clojure -A:depstar -m hf.depstar.jar northstar-model-0.0.1.jar.
The consume the library artifact from other local projects, I take it I manually install to my ~/.m2/ from here -- i.e. copying pom.xml & northstar-model-0.0.1.jar to ~/.m2/northstar/model/0.0.1/ #2019-09-05 23:05:36 seancorfield @quest If you have mvn installed, that's probably the easiest way to do it "right". #2019-09-05 23:06:32 seancorfield (`mvn` is what I use to deploy JARs built with depstar to http://clojars.org -- I've never tried to to install locally but I'm pretty sure it's a similar command) #2019-09-05 23:09:24 seancorfield mvn install:install-file -Dfile=northstar-model-0.0.1.jar -DpomFile=pom.xml
I believe #2019-09-05 23:10:28 seancorfield #2019-09-05 23:14:13 Quest Hmm, I'll keep looking through maven install & see if I can find something that works. Faking the .m2 autogen files isn't working out #2019-09-05 23:15:15 seancorfield @quest The above command seemed to work for me, based on limited testing #2019-09-05 23:17:01 Quest Yes! I can confirm that it's working. Thanks @seancorfield
clojure -A:depstar -m hf.depstar.jar northstar-model-0.0.1.jar
mvn install:install-file -Dfile=northstar-model-0.0.1.jar -DpomFile=pom.xml
;; include dependency [northstar/model "0.0.1"]
#2019-09-05 23:17:26 Quest It seems like it'd be good to document this somewhere, but I'm not exactly sure where 🤔 #2019-09-05 23:18:01 seancorfield I should add it to the readme... #2019-09-05 23:18:45 Quest I'll open a PR with my best attempt to capture it, edit it as needed from there? 🙂 #2019-09-05 23:19:01 seancorfield Sure! Thank you! #2019-09-05 23:19:23 seancorfield Deploy to Clojars is mvn deploy:deploy-file -Dfile=${1}.jar -DpomFile=pom.xml -DrepositoryId=clojars -Durl= #2019-09-05 23:20:35 seancorfield (where ${1} is the jar file name -- and this assumes your credentials are in ~/.m2/settings.xml like <settings>
<servers>
<server>
<id>clojars</id>
<username>someperson</username>
<password>topsecret</password>
</server>
</servers>
</settings>
#2019-09-06 15:43:32 Travis I may have over looked this in the docs but is there something kind of the equivalent of lein deps
. Basically trying to make sure all maven deps are downloaded for caching #2019-09-06 15:55:45 seancorfield @camechis use clojure -Spath or clojure -Stree (and just ignore the output). #2019-09-06 15:56:05 Travis sweet, thnx @seancorfield #2019-09-06 16:49:24 jjttjj @quest this might not fit your use case but if you're just looking to use code from one of your local projects it's dead easy to just refer to local source directories with clj: :deps {
...
{:local/root "/home/jjttjjj/code/goodlib"}
...
} #2019-09-06 16:50:56 alexmiller you should use :paths for that #2019-09-06 17:50:09 Quest @jjttjj Unfortunately my other projects aren't deps.edn yet -- adapting the re-frame 10x (shadow-cljs) template isn't something I'm ready for yet. Will keep that in mind though #2019-09-07 02:15:02 seancorfield seancorfield/depstar "0.3.3" is available -- no longer uses clojure.xml so you won't get "Illegal Reflective Access" warnings on JDK11 any more! #2019-09-07 18:32:59 tianshu I'm trying to create my own clj project template using clj-new. when using it, there's an error:
Failed with: Failed to resolve version for :home/tianshu/develop/ministack/lein-template:jar:RELEASE: Could not find metadata :home/tianshu/develop/ministack/lein-template/maven-metadata.xml in local (/home/tianshu/.m2/repository)
Execution error (ExceptionInfo) at clj-new.helpers/resolve-remote-template (helpers.clj:148).
Could not load artifact for template: /home/tianshu/develop/ministack
Tried coordinates:
[/home/tianshu/develop/ministack/boot-template "RELEASE"]
[/home/tianshu/develop/ministack/lein-template "RELEASE"]
Full report at:
/tmp/clojure-14768781517611096608.edn
what does this mean, why a lein-template and boot-template is needed? #2019-09-07 18:36:46 hiredman you are somehow declaring a maven dependency with a group-id of /home/tianshu/develop/ministack/ and an artifact-id of lein-template #2019-09-07 18:36:54 hiredman which is obviously bogus #2019-09-07 18:38:11 tianshu But it is said I should use qualified symbol or multi-segment clojure symbols.
Project names must be valid qualified or multi-segment Clojure symbols. #2019-09-07 18:38:48 tianshu Ohh #2019-09-07 18:38:51 tianshu I know #2019-09-07 18:38:52 tianshu sorry #2019-09-07 18:40:30 tianshu I give it the wrong template path #2019-09-07 18:41:17 seancorfield clj-new looks for a clj template, then a boot template, then a lein template -- so you can use it to create projects with other tools' templates. #2019-09-07 18:42:23 seancorfield (it's based on boot-new which looks for a boot template, then a lein template -- and I based that library in turn on the innards of lein new, and in fact both boot-new and clj-new include some Leiningen code, with permission) #2019-09-07 18:47:00 tianshu thanks for the explain! #2019-09-07 18:48:05 seancorfield The idea behind "Project names must be valid qualified or multi-segment Clojure symbols." is to avoid the everything.core default that Leiningen (and Boot) fall back on if you provide a single segment name. #2019-09-07 18:52:09 seancorfield For qualified names, it sort of expects you to use something relatively unique like your GitHub username when creating a project: clj -A:new some-template tianshu/example which will create a folder called example but the main namespace in that project would be tianshu.example #2019-09-07 18:53:34 seancorfield Of course you can just go ahead and produce the same thing as Leiningen: clj -A:new some-template example.core 🙂 The folder will be example.core but the main ns will be example.core from src/example/core.clj ... ugh! #2019-09-07 18:55:23 seancorfield And finally if you combine them: clj -A:new app tianshu/example.core you'll get a top-level example.core project folder with src/tianshu/example/core.clj for a main ns of tianshu.example.core #2019-09-08 07:45:29 tianshu yeah, make sense. thanks very much! @seancorfield #2019-09-08 22:10:29 cfleming I’m adding code to Cursive to cache project dependency resolution, and also classpaths for things like running REPLs. Here’s what I’m hashing to create a cache key:
1. Global config (whether using CLI or t.d.a directly, and identifying properties like install path & version in each case), and user and system (if applicable) deps.edn files.
2. Hash of all non-whitespace non-comment content from project deps.edn, and all transitive deps.edns reachable via :local/root.
3. Aliases in use, depending on operation. #2019-09-08 22:11:33 cfleming In the case of the execution classpath, I’ll also add in details from the execution, e.g. main namespace, env vars etc, basically all the info from the run configuration. #2019-09-08 22:11:42 cfleming Is there anything else that might affect that I’ve forgotten? #2019-09-08 22:20:59 seancorfield What about the user-level deps.edn which may include aliases/extra-deps/override-deps? #2019-09-08 22:22:33 seancorfield At World Singles Networks, we use CLJ_CONFIG to set the "user-level" to be our repo-wide deps.edn and then each subproject has a deps.edn file. Those two combined then form the set of deps and aliases etc for each given subproject. We use the repo-wide ("user-level") deps.edn file to "pin" versions by having a :defaults alias with all our :override-deps in. #2019-09-09 00:16:01 cfleming @seancorfield Ah, yes, thanks - I actually include that already in option #1 #2019-09-09 00:16:38 cfleming Although Cursive doesn’t yet support CLJ_CONFIG - that’s on my TODO list #2019-09-09 00:18:11 cfleming I’ve edited the list above to make that clear. #2019-09-09 00:19:00 cfleming Those files include the same non-whitespace non-comment content logic. #2019-09-09 00:23:26 alexmiller did you look at the clojure script? you can see everything included in the hash now #2019-09-09 00:24:08 alexmiller seems like matching that is best for matching behavior #2019-09-09 00:25:09 seancorfield @alexmiller Did you say CLJ_CONFIG support might go away? Is there an alternative in mind where a (large) project/monorepo may want a "base" deps.edn file and then to have a separate deps.edn for each subproject? #2019-09-09 00:25:43 alexmiller no, I didn't say that #2019-09-09 00:25:59 seancorfield Ah, then I perhaps misunderstood you 🙂 #2019-09-09 00:26:01 alexmiller more likely to add something to insert more #2019-09-09 00:26:52 seancorfield OK. Having four deps.edn files being considered would be fine too. System, user, (project/repo -- new!), (sub)project/current dir. #2019-09-09 00:27:22 seancorfield (I mostly don't care about the first two in this case -- I really care about the "third" and fourth 🙂 ) #2019-09-09 00:29:03 cfleming I did look at the script, but a while ago. IIRC it doesn’t include deps file contents, only the config paths, and definitely not the contents of transitive local roots. #2019-09-09 00:29:29 cfleming At least, I couldn’t see where the deps file contents were taken into account. #2019-09-09 00:33:57 alexmiller yeah it's just the paths and aliases and the -Sdeps contents #2019-09-09 00:34:38 cfleming Actually, does the -nt test take the file timestamps into account too? My google-bash-fu is failing me. #2019-09-09 00:34:49 alexmiller "newer than" #2019-09-09 00:35:12 cfleming Right, that was what I suspected, but Google wasn’t very helpful. #2019-09-09 00:35:14 alexmiller so the "stale" calculation is separate from the hash #2019-09-09 00:35:33 cfleming Right. #2019-09-09 00:36:07 cfleming In my case I don’t need a location (i.e. what you’re using the hash for), I just need the staleness test. #2019-09-09 00:36:46 alexmiller so yeah, that's largely timestamp based #2019-09-09 00:37:50 cfleming Ok, thanks. In my case it’s relatively easy to do the full transitive contents check, so I’ll stick with that. #2019-09-09 00:37:51 alexmiller well, and assuming the hash finds there is a cp to use #2019-09-09 00:38:03 cfleming Right. #2019-09-09 00:43:41 cfleming Actually, one other question I had: since the system deps file is now read from a resource, am I correct in thinking that for recent versions I only need to pass the user config, and not both? #2019-09-09 00:44:27 cfleming I looked at the code a while back, and it looked like the resource will always be put first in the list, so if I also pass the system deps file then that will override the one from the resource, but will presumably be the same? #2019-09-09 00:45:40 cfleming Is there any way to tell from e.g. -Sdescribe whether I need to pass the system file or not? The path still appears in the config-files list. #2019-09-09 00:48:03 cfleming If not I can just check the version and choose based on that. #2019-09-09 00:52:54 seancorfield @cfleming Looks like you could check :install-dir against the first path in :config-files -- but I will note that t.d.a. has an API for getting the deps.edn files that returns just the user + project level ones (because it has the system one baked in). #2019-09-09 01:07:27 cfleming Interesting. It’s potentially tricky to call that API ahead of time when delegating to the CLI. #2019-09-09 01:07:40 cfleming I’ll take a look at that though, thanks. #2019-09-09 02:06:04 alexmiller it wouldn't hurt to include both #2019-09-09 03:04:40 cfleming Is there a reason that t.d.a and tools.gitlibs print progress information (“Downloading: xxx” and “Checking out: xxx”) to stderr? #2019-09-09 03:20:46 alexmiller so you can shunt it off if you don't want to see it #2019-09-09 03:22:03 alexmiller normally it's silent on stdout unless you do something that expects a printed result, like -Stree or -Spath etc #2019-09-09 04:07:09 cfleming Ah, ok, thanks. #2019-09-10 16:37:14 jjttjj Is there any obvious reason I'd get the message
Skipping paths: resources
after running clj -Spom when resources is in my :paths? #2019-09-10 16:38:19 alexmiller it means it saw you had that in paths, but there is no canonical way to do that in pom.xml so, it just didn't do anything with it #2019-09-10 16:39:01 alexmiller in Maven, you typically need to pull in a plugin and do several steps to work with a resources dir #2019-09-10 16:40:01 jjttjj strange, I somehow managed to deploy to clojars on a different computer (with depstar+maven), and don't remember any issues getting resources in the jar #2019-09-13 15:48:22 plexus I'm having some trouble using a jar from a private Maven repo (we're using Gitlab's built-in maven repos) #2019-09-13 15:49:19 plexus I can install it fine with Maven, and see it in ~/.m2, but when I do clj it seems it still tries to fetch it, Error building classpath. Could not find artifact group:artifact:jar:version in central () #2019-09-13 15:49:40 plexus is this to be expected, or am I holding it wrong? #2019-09-13 15:56:52 dominicm I think it will try there, but it should also try from your repo that's configured #2019-09-13 20:25:00 vlaaad Couple of notes:
- there is java 8 by default, so if you don't need a newer one you can omit actions/ action
- use clojure instead of clj, duh #2019-09-13 20:27:43 vlaaad I also want to say again how much I love tools-deps. Aliases and their simplicity is amazing: adding workflow-specific aliases is such a breeze, and mixing them in different contexts is very straightforward. Thank you very much cognitect and @alexmiller #2019-09-13 20:30:54 vlaaad I have :test alias to add test directory and test-specific deps, I have :runner alias to run tests, I have :headless alias to run cljfx without display. During dev I have :test alias on, I can run tests locally with -A:test:runner, and I can make CI run tests in headless mode with -A:test:runner:headless. So simple clj #2019-09-13 20:31:09 gerred hey! as someone doing a lot with github actions too, this is great. I might recommend using some more Docker-based actions rather than shell-based ones, which might make things a little bit faster. #2019-09-13 20:38:33 vlaaad But then I'd need to figure out docker in addition to github actions, I'm too lazy 😛 #2019-09-17 15:23:02 avi Totally understandable Vlad, but if and when you do want to give it a try, I think GitHub has done a good job making this super simple. Basically you just add a container key to your job definition and Actions takes care of getting the image, starting a container, and then running your job’s steps within that container.
So for example you could add container: clojure:tools-deps to your job definition and then in your steps you wouldn’t need to set up Java nor install Clojure. Presumably this would become faster over time as GitHub does a better job of caching docker image layers. And there are a bunch of variants of the clojure image, e.g. with different OS’s, different JREs, etc.
HTH but feel free to ignore if it’s not the right time! #2019-09-17 15:27:17 gerred :+1: also happy to help 🙂 #2019-09-17 15:29:41 gerred it'll even build a repo from a Dockerfile, which adds time but is great for development #2019-09-17 15:30:12 avi Ooo really? I didn’t know that — that’s excellent! I gotta look up the docs on that. Thanks! #2019-09-17 15:30:41 gerred of course. if you use actions/checkout and actions/docker, it's all Dockerfile based so there's pre-steps to build those images. #2019-09-17 15:31:42 gerred "Build x..." aren't steps in our workflow #2019-09-17 15:31:47 gerred they just happen #2019-09-17 15:32:27 avi Aha, that makes sense! It’s kinda naturally intuitive once I unlearn the constraints I learned from CircleCI, wherein for a given job I have to choose between a VM and a container, so there’s no concise way to build an image and then use it. #2019-09-17 15:32:30 avi Thanks! #2019-09-17 15:33:33 gerred of course. #2019-09-17 15:34:38 gerred i'm also well integrated with the actions team and the implementation since we're doing a lot kubernetes-wise with this, so if you need any help or support, let me know and I can either help or point you at the right people within github. #2019-09-17 15:41:48 avi Will do — thanks! #2019-09-17 15:42:25 vlaaad Hey, many thanks for the info! I have a question: what problem does docker solve that actions don't? You mentioned that you can have different docker images with different OSs, JREs — isn't it something already solved by actions matrix? #2019-09-17 15:43:44 gerred actions in-and-of-themselves are either based on Docker or javascript. #2019-09-17 15:43:53 gerred workflows are made up of a series of actions, which build up matrices, etc. #2019-09-17 15:44:07 avi I’m not sure about actions matrix; I haven’t looked closely at it yet. As for using docker in general, it can sometimes make jobs more concise and sometimes, theoretically, faster, because the images might be cached. #2019-09-17 15:44:28 gerred you can also technically just run shell commands on the host runs-on I guess. #2019-09-17 15:44:36 gerred brb, doing a CNCF TOC presentation, I'll be back in a bit. #2019-09-17 15:44:45 avi It can also make it easier to debug/troubleshoot issues in a CI job because you can then use the same exact environment locally to reproduce the situation #2019-09-17 15:46:03 vlaaad Ah, you mean if I have docker setup at local machine #2019-09-17 15:46:20 avi Yup #2019-09-17 15:46:33 vlaaad Got it, thanks! #2019-09-17 15:46:51 gerred and you can think of an action as an individual step (but reproducible as a lego block) in a whole workflow #2019-09-17 15:46:54 gerred with the v2 actions API #2019-09-13 20:31:31 gerred (I have a clj-kondo action currently in progress as well) #2019-09-13 20:40:41 vlaaad What I'd really like to have is a tools-deps action, with .m2 and .gitlibs caching etc. Configurable by aliases and deps.edn in a project. #2019-09-13 20:32:13 gerred this is great though! #2019-09-17 19:29:27 avi 👋 are there any known gotchyas with using gitlibs via t.d.a in CircleCI? I just switched a lib from Maven coords to git coords; it worked fine locally for me but over in CircleCI the deps were downloaded but then the files couldn’t be found on the classpath during the test run. #2019-09-17 19:37:22 ghadi @aviflax hard to say without seeing what bin/tests-with-coverage is doing #2019-09-17 19:37:40 ghadi in general if you have private ssh repos, you'll need to use an ssh-agent #2019-09-17 19:37:44 avi clojure -J-Xmx2g -A:test:test/coverage
#2019-09-17 19:37:49 avi It’s not a private repo #2019-09-17 19:38:17 avi I was able to reproduce the problem just now locally via Docker so I should 😅 be able to figure it out #2019-09-17 19:46:10 avi It looks like the problem is that when the classpath is generated, there’s an extra segment main at the end; that dir doesn’t exist in the repo hosting this particular library #2019-09-17 19:46:29 avi Here’s the cp entry: /home/circleci/.gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/58a396f54f469e98b5aae92b9357d65434914228/src/main #2019-09-17 19:48:38 ghadi Seems like that lib has an invalid paths inside its deps.edn? #2019-09-17 19:48:59 avi ah, maybe, I’ll take a look #2019-09-17 19:49:21 avi Nope #2019-09-17 19:53:34 ghadi It needs to specify paths explicitly #2019-09-17 19:54:17 avi Ohh wait I remember this #2019-09-17 19:54:25 ghadi :paths ["src"]. I think there is a bug when it isn't set #2019-09-17 19:54:29 avi the dep is inheriting the source path that was specified in my project root #2019-09-17 19:54:38 ghadi Yup #2019-09-17 19:54:41 avi I mean, my project’s deps.edn #2019-09-17 19:54:46 avi IIRC this is a known bug #2019-09-17 19:55:10 avi I’ll see if I can find the JIRA issue #2019-09-17 19:55:24 dominicm Make sure you vote! #2019-09-17 19:56:23 avi will do, if/when I find it #2019-09-17 19:56:27 avi so far no dice #2019-09-17 19:56:53 vlaaad wasn't it fixed? #2019-09-17 19:58:02 avi It must be that the Docker image I’m using hasn’t picked up the fix yet #2019-09-17 19:58:19 avi I’m using circleci/clojure:openjdk-11-tools-deps-browsers #2019-09-17 19:58:50 avi @ghadi you reported that bug! #2019-09-17 19:59:38 ghadi It's the circle of life #2019-09-17 20:01:42 avi 😂 #2019-09-17 20:01:48 avi Thanks for the help! #2019-09-17 20:07:54 avi Alex commented on the issue:
> Released in clj 1.10.1.458
And I’ve confirmed that the CircleCI image I’m using has 1.10.0.442 installed.
Interestingly, even the official Docker images for Clojure don’t appear to have any variant with a version of Clojure more recent than 1.10.0.442. This is surprising… #2019-09-17 20:10:11 alexmiller there are no official Docker images for Clojure #2019-09-17 20:11:43 gerred i don't even think there should be official Docker images for Clojure in a world of uberjars, to get edgy #2019-09-17 20:11:27 avi ah, well that explains it #2019-09-17 20:11:54 gerred that's Docker owned, like the company. #2019-09-17 20:12:21 avi So if I think they should be updated, I guess I pester Docker, the company 😬 #2019-09-17 20:12:24 avi OK thanks all! #2019-09-17 20:12:50 gerred yup! if you're interested in custom images and interested in sponsoring some sort of support model, I'd love to chat though. #2019-09-17 20:13:00 gerred we made a #cloud-native channel lately. #2019-09-17 20:13:17 gerred so I'm in a process of taking in problems/opinions and trying to distill something #2019-09-17 21:11:35 avi Next question:
• I’ve been using Cambada to AOT my code and build an uberjar containing my code and all my dependencies.
• Now that I’ve switched one of my dependencies to a gitlib, Cambada is having trouble building the uberjar.
So: Does anyone happen to know whether there’s an existing solution to this somewhere?
(I mean either some way to continue using Cambada, maybe a fork, etc; or some alternative to Cambada (whether a single tool or a combination of tools) that I might want to consider switching to.) #2019-09-18 13:23:52 avi Will do — thanks! #2019-09-18 15:58:19 avi @U097654L8 it doesn’t seem to work; it seems to fail with the same error as the root cambada.
I see you have GitHub’s issues feature disabled in your fork but the feature is enabled in the root repo — would you be interested in a bug report opened as an issue in that repo, which we could use to discuss and debug the problem, and to potentially craft a fix to be merged into your fork? #2019-09-18 16:24:57 avi @U097654L8 never mind, I spoke too soon — I took a closer look at the error output and it was not the same error — it was a different error that I’ve seen before and already have a way to work around. And once I applied that workaround, the uberjar process completed successfully. So your fork does in fact resolve my issue with gitlibs. This is great — thank you! #2019-09-18 18:33:32 avi For anyone who might have a similar question: @U097654L8 suggested I try this fork of Cambada: https://github.com/mikeananev/cambada — and it does indeed seem to have fixed the problem I was encountering with gitlibs, AOT, and uberjarring. Thanks Mike! #2019-09-17 21:43:18 seancorfield @aviflax Is AOT a requirement? #2019-09-17 21:44:24 avi I guess it’s not a 100% hard requirement but I’d prefer to preserve it. The project is a CLI tool so it needs to start quickly. #2019-09-17 21:45:17 avi Thanks, I was just reviewing that list. I was just wondering if anyone had an answer off the top of their head. The descriptions of those tools don’t always include lots of details. #2019-09-17 21:45:24 seancorfield I don't know how many of those do AOT although, in theory, you could always do that step separately and then package stuff: -e "(compile 'my.main.ns)" #2019-09-17 21:46:27 seancorfield I am fairly certain that depstar works with git deps and local deps but it does not do AOT. #2019-09-17 21:46:40 avi I believe Cambada is AOT compiling all the Clojure code in my project, including in dependencies. I like that. #2019-09-17 21:47:04 avi Would calling compile do the same? I’d guess not... #2019-09-17 21:47:15 seancorfield Since AOT is transitive, I suspect just compiling the main ns will get you much the same result...? #2019-09-17 21:47:44 avi It’s worth trying out for sure! #2019-09-17 21:47:59 avi Gotta get on my bike now, will check back in shortly. Thanks! #2019-09-17 21:48:16 seancorfield You'll need to ensure classes exists as a top-level folder in your repo and add it to the :paths vector so it's on the classpath. #2019-09-18 00:25:42 cfleming Has anyone made a spec for deps files that I could use to check validity? #2019-09-18 00:26:04 cfleming It’s probably not hard to put together, but I wasn’t sure if anything already existed. #2019-09-18 00:26:09 alexmiller I think there are some in tools.deps #2019-09-18 00:26:32 alexmiller but I don't actually use them atm, so they might not be totally up to date (but I'd be happy to fix that) #2019-09-18 00:26:46 cfleming Ok, I’ll check them out, thanks. #2019-09-18 00:27:42 cfleming I’d like to make at least a basic check that the file looks good before automatically trying to resolve it on file modification. #2019-09-18 00:27:44 alexmiller ::deps-map in particular #2019-09-18 00:28:27 cfleming You think ::deps-map might not be up to date? #2019-09-18 00:28:42 alexmiller just eyeballing it, it can't be too far off #2019-09-18 00:29:06 cfleming I’m mostly interested validating the input, i.e. the deps file itself, not what t.d.a returns. #2019-09-18 00:29:27 alexmiller yeah that ::deps-map is a deps.edn file contents #2019-09-18 00:29:36 cfleming Ah, ok. #2019-09-18 00:31:10 alexmiller I'm pretty sure I'm not using those specs in the mainline code right now so it's probably as good as any code you aren't using #2019-09-18 00:31:56 cfleming Ok. I might have a go at converting this to my internal validator, which means I could show errors in the editor too. #2019-09-18 00:32:30 cfleming As well as checking before trying to resolve automatically. #2019-09-18 00:33:01 alexmiller I don't see anything wrong in those specs. it's missing :sha and maybe one or two other things #2019-09-18 00:33:26 cfleming Ok, if I see any obvious errors I’ll make a PR #2019-09-18 00:33:35 cfleming Or a patch, rather. #2019-09-18 00:34:10 alexmiller or just tell me and I'll fix it :) #2019-09-18 00:34:20 cfleming These days, when supplying a patch, does it have to be attached to a JIRA? Or can it be attached to http://ask.clojure.org? #2019-09-18 00:34:25 cfleming Or that, yes 🙂 #2019-09-18 00:35:33 alexmiller as someone that's supposed to be writing a talk right now, I'm in the market for sources of procrastination :) #2019-09-18 00:35:56 cfleming Meh, you still have two weeks, what’s the issue? #2019-09-18 01:11:20 alexmiller one week!! #2019-09-18 02:08:26 cfleming What are plane flights for? #2019-09-18 00:39:06 cfleming This is probably more of a question for #spec, but I’m curious about this:
(s/def :local/coord (s/keys :req-un [:local/root] :opt-un [::path]))
#2019-09-18 00:39:34 cfleming Does that mean that :req-un can be used for namespaced keywords as long as the ns is explicitly specified? #2019-09-18 00:41:38 cfleming Ah, no, ignore me - never mind. #2019-09-18 00:42:31 cfleming Actually, no - the doc says:
> These variants specify namespaced keys used to find their specification, but the map only checks for the unqualified version of the keys. #2019-09-18 00:42:53 cfleming Does that mean that the spec would accept :root as well as :local/root? #2019-09-18 01:02:34 alexmiller ha, I was off checking many things and independently discovered this - it's a bug #2019-09-18 01:02:52 alexmiller I ran it on all the deps.edn in my system and that barfed at me in a few places #2019-09-18 01:09:10 alexmiller fixed and pushed #2019-09-18 01:09:16 alexmiller also added the required :sha attr for git coords #2019-09-18 02:00:40 alexmiller released as tda 0.7.549 if you need it #2019-09-18 02:08:48 cfleming Great, thank you! #2019-09-18 03:17:39 Ahmed Hassan What is :resource-paths (leiningen) counterpart in Tools Deps? #2019-09-18 03:17:52 Ahmed Hassan Is it :paths? #2019-09-18 03:48:57 cfleming @ahmed1hsn Yes - Deps doesn’t make a distinction. #2019-09-18 12:47:29 alexmiller to be more accurate, the jvm doesn't make a distinction, and thus neither does deps #2019-09-18 13:19:55 dominicm I stopped making the distinction in my programs too. I realised it was rather arbitrary. #2019-09-20 22:05:58 shaun-mahood For clj-on-windows, how would I start a socket repl? I'm trying powershell -command clj '-J"-Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}""' but not sure if I have the quotes wrong or if there is something else going on. The REPL starts up, but I can't get to the socket repl at all. #2019-09-20 22:08:56 seancorfield @shaun-mahood Did you try with comma instead of space in that? #2019-09-20 22:09:18 shaun-mahood Of course not - I never think of the Corfield comma 🙂 #2019-09-20 22:09:34 seancorfield Alternatively, just use an alias in your deps.edn (which is how I do it on Powershell). #2019-09-20 22:10:32 seancorfield (and it has to have commas in the deps.edn version because of the round-tripping through the .cpcache files and shell etc) #2019-09-20 22:15:16 seancorfield @shaun-mahood Did it work with the comma? #2019-09-20 22:16:52 shaun-mahood No luck yet - I might be putting the commas in the wrong place though #2019-09-20 22:18:07 shaun-mahood I just realized I have to put lein on that machine, though, so I might use that instead - saves me from figuring out how to get Cursive to connect to a socket repl #2019-09-20 22:19:49 seancorfield Cursive requires nREPL doesn't it? So you wouldn't want to start a Socket REPL anyway. #2019-09-20 22:21:07 seancorfield clj -A:nrepl 🙂 #2019-09-20 22:21:43 seancorfield :nrepl {:extra-deps {nrepl {:mvn/version "RELEASE"}}
:main-opts ["-m" "nrepl.cmdline"]}
#2019-09-20 22:22:32 shaun-mahood Yeah that looks nice and easy 🙂 #2019-09-20 22:23:49 seancorfield This is why I have so much stuff in my .clojure/deps.edn file -- and I have it on every machine and under every shell (macOS, Windows Powershell, WSL, etc) #2019-09-20 22:54:12 shaun-mahood @seancorfield It looks like it was a firewall issue, not a Clojure issue - thanks for the help, using your aliases I at least found where the problem was. Now to figure out why Windows Server is ignoring me when I tell it to open a port... #2019-09-25 14:38:18 val_waeselynck Is it possible to deps-consume a JAR that's versioned in a Git repo?
Context: I have a little private java library that I want to consume in a deps.edn project. Putting the generated JAR under version control is weird, but this seems to be a minor inconvenience compared to setting up a private Maven distribution infrastructure. #2019-09-25 14:50:20 dominicm If something was going to work, it would be :deps/root #2019-09-25 14:54:02 val_waeselynck Like this?
{:git/url " #2019-09-25 14:54:33 val_waeselynck (Assuming the JAR was generated at the root of the project). #2019-09-25 20:09:44 alexmiller That won’t work #2019-09-26 05:36:26 val_waeselynck Thanks @U064X3EF3. Do you see any other way? #2019-09-26 07:27:54 alexmiller you could add the jar file to your :paths in the deps.edn of the git dep #2019-09-26 11:04:21 dharrigan Hello! Could master of tools.deps.alpha be merged into add-libs please? The add-libs is of tools.deps.alpha-0.7.541 (August 9th), not tools.deps.alpha-0.7.549 (September 17th)? #2019-09-26 11:04:49 dharrigan Thank you very much 🙂 I guess, any reason why add-lib can't be brought in to master? It's super useful 🙂 #2019-09-26 11:11:51 alexmiller I don’t think you’re missing anything from master - just some tweaks in the specs (which aren’t used) #2019-09-26 11:12:24 dharrigan fantastic! 🙂 Thank you (for clarifying!) #2019-09-26 11:13:40 alexmiller There are issues with add-lib in some environments and it’s unlikely to end up included in its current form (some of it may actually be moved into Clojure itself) #2019-09-26 11:14:18 dharrigan kk, that's good to know. Havin the ability to dynamically pull in a dep - for experimentation on the repl, is amazingly useful to have 🙂 #2019-09-26 11:14:34 dharrigan (and given clojure's great interop, doing that playing around is a very happy process!) #2019-09-26 12:14:42 dominicm Why move it into clojure @alexmiller? Which bits? #2019-09-26 13:01:30 alexmiller the part that tracks the current “libs” currently in use could be built in. It’s already designed to be initialized in a couple ways so even non tools.deps tools could convey the lib set via those means. add-lib itself relies on tools.deps so that probably cannot move, but the parts that update the lib set could be genetically useful #2019-09-26 13:02:38 alexmiller Generically :) #2019-09-26 13:06:40 dominicm What's the goal of making the concept of a library first class? I can imagine using it to affect requires in different ways (e.g. Namespace versioning). Is there a set of goals in mind yet? #2019-09-26 13:08:06 alexmiller No, none of this is baked which is why it’s on a branch #2019-09-26 13:56:20 alexmiller There are a lot of potentially interesting things that could be done though #2019-09-27 15:27:56 plexus seems Clojure CLI uses a jar that contains Clojure + tools.deps. How would I build such a jar myself? or put in a different way, how can I make clojure use my local changes to tools.deps to try them out? #2019-09-27 15:39:35 alexmiller The easiest way (what I do) is to just hack my clojure script to put my local source directory at the front of the tools classpath #2019-09-27 15:39:48 alexmiller Then you can pick up changes with no build at all #2019-09-30 04:08:28 taylor I have a deps.edn project/repo that I’d like to also keep example projects, but ideally those wouldn’t get pulled when another deps.edn project depends on mine e.g. via git. Would tools.deps pull those “extra” files from my repo, and the README and everything? Should I stop worrying about a few kilobytes? #2019-09-30 04:14:53 seancorfield @taylor If someone depends on your project via git deps, the whole repo will be checked out (regardless of which parts are actually used). #2019-09-30 04:15:39 seancorfield If you publish an artifact (on Clojars or wherever) then you can choose what goes into that. #2019-09-30 06:43:00 sogaiu you get a bare clone of the repository + one or more checkouts corresponding to whatever commits are used, right? at least, that's how it looks under ~/.gitlibs/ #2019-09-30 12:49:16 alexmiller Yes, that’s correct #2019-10-01 00:36:43 plexus If there are gitlab users here that would be interested in using the private Maven repositories provided by gitlab, then please check out this patch and provide feedback. https://clojure.atlassian.net/browse/TDEPS-140 #2019-10-01 00:38:03 plexus Gitlab uses a token in a http header for authentication instead of the normal username/password method, this patch adds support for that. #2019-10-01 04:17:16 mattyulrich is it possible to resolve a pom type dep (rather than a jar) in my deps.edn file? #2019-10-01 04:21:01 mattyulrich I think in lein, one could specify :type "pom" and it would resolve the pom alone without requiring a jar at the end... #2019-10-01 04:29:56 mattyulrich oh.. nevermind - I just found the :extension "pom" attribute - did the trick perfectly! #2019-10-01 12:17:25 kirill.salykin Hi
I am playing with Github actions and tools.deps, seems there is an issue with installing deps from private github repositories
Did anyone also experienced it? Do I need to install my ssh key? #2019-10-01 12:23:11 alexmiller Yes, you would need to do that #2019-10-01 12:23:17 kirill.salykin k, thanks #2019-10-01 12:23:45 alexmiller I believe there is a special env var for it automatically in the action #2019-10-01 12:24:10 alexmiller But you’d need to add it to the ssh-agent #2019-10-01 12:24:34 alexmiller I haven’t done this yet myself so don’t have a recipe for you #2019-10-01 12:24:49 kirill.salykin thanks, will try to install ssh key and add it to agent #2019-10-01 12:59:00 kirill.salykin adding ssh key didnt help much
name: CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/ #2019-10-01 12:59:19 kirill.salykin Error building classpath. connector is not available:
com.jcraft.jsch.agentproxy.AgentProxyException: connector is not available:
at com.jcraft.jsch.agentproxy.ConnectorFactory.createConnector(ConnectorFactory.java:120)
#2019-10-01 12:59:31 kirill.salykin please advice what can be wrong? #2019-10-01 14:34:39 ghadi @kirill.salykin in Github Actions, do processes persist between steps? #2019-10-01 14:42:46 kirill.salykin I don’t know, to be honest
testing now #2019-10-01 14:55:24 kirill.salykin yes, you are correct #2019-10-01 14:55:37 kirill.salykin run: eval $(ssh-agent -s) && ssh-add -k ~/.ssh/id_rsa && ./bin/run-tests-ci.sh
this works #2019-10-01 15:02:45 kirill.salykin > My guess is eval $(ssh-agent -s) dies at the end of that step?
this is my guess as well #2019-10-01 14:35:03 ghadi You can find out by running
- run: ssh-add -l
in a subsequent step #2019-10-01 14:35:23 ghadi I have github actions building private git deps, but I'm doing it all within a container #2019-10-01 14:39:35 kirill.salykin i am able to clone that repo via just git #2019-10-01 14:40:06 ghadi but tools.deps isn't using git #2019-10-01 14:40:52 kirill.salykin yeah, just to ensure that I can clone it #2019-10-01 15:02:05 ghadi My guess is eval $(ssh-agent -s) dies at the end of that step?
@kirill.salykin #2019-10-01 15:02:23 ghadi oh just saw the thread #2019-10-01 15:04:10 ghadi you should probably pin the https://github.com host key statically in source code #2019-10-01 15:04:24 ghadi otherwise it will accept any known host key -- MITM attack #2019-10-01 15:04:31 kirill.salykin true, thanks #2019-10-03 16:58:06 dominicm @alexmiller bit of a weird one https://github.com/juxt/pack.alpha/issues/64 no repro yet exactly, but it looks like RepositorySystem is coming back as nil. I don't suppose you're clued up on this and can think of an obvious cause? #2019-10-03 17:28:29 alexmiller I looked at that when it was filed as it was mentioned here too. It's weird, don't know what to make of it. I think from the stack this pre-dates the session cache additions? #2019-10-03 17:30:24 dominicm It's still happening after updating too. #2019-10-03 17:30:54 dominicm Digging in, the impl of the service locator does attempt to add the repository system, so that should be there. #2019-10-03 17:31:41 dominicm Bumping from 1.1.1 of the maven dependencies to 1.4.1 doesn't fix it either. #2019-10-03 17:37:45 dominicm this is the problem with service locators. All the damn coupling. #2019-10-03 17:57:13 Joe Lane Hey friends, I'm having an issue cloning my own private github repo with tools deps. Even though I'm able to clone the repo with the ssh url into a local tmp directory, JGit is still telling me that Caused by: org.eclipse.jgit.errors.TransportException: .
I'm on a mac with these version numbers:
Joes-MacBook-Pro:snapestry-ion magemasher$ clj -Sverbose
version = 1.10.1.469
install_dir = /usr/local/Cellar/clojure/1.10.1.469
config_dir = /Users/magemasher/.clojure
config_paths = /usr/local/Cellar/clojure/1.10.1.469/deps.edn /Users/magemasher/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/479583893.cp
The ssh key I have on github matches this key below:
ssh-add -l -E md5
4096 MD5:10:ef:06:6b:9b:94:86:fc:06:83:9f:17:0a:8c:bb:10 /Users/magemasher/.ssh/id_rsa_jplane (RSA)
Does anybody have any ideas? I'm really stumped here. #2019-10-03 17:57:42 Joe Lane #2019-10-03 17:58:32 alexmiller is ssh-agent running? #2019-10-03 17:59:25 Joe Lane yes #2019-10-03 17:59:39 Joe Lane (I just confirmed by starting it, same failure) #2019-10-03 18:00:33 alexmiller there are some known failure modes with host key algorithms etc, but this doesn't really look like that #2019-10-03 18:00:42 alexmiller I assume you can git clone with that key #2019-10-03 18:00:47 Joe Lane yep #2019-10-03 18:01:09 ghadi Do you have entries in .ssh/config ? #2019-10-03 18:01:26 ghadi Sometime jsch tries to read those instead of the ssh agent #2019-10-03 18:02:03 Joe Lane I do, are they safe to paste? #2019-10-03 18:02:09 ghadi Yes #2019-10-03 18:02:26 ghadi It’s usually just file names #2019-10-03 18:02:32 alexmiller there's one like IdentityKey or something that might need to be removed #2019-10-03 18:02:45 Joe Lane Host *
AddKeysToAgent yes
IdentityFile ~/.ssh/id_rsa_jplane
IdentitiesOnly yes
Host
HostName
User admin
IdentityFile ~/.ssh/id_rsa_jplane
#2019-10-03 18:03:01 alexmiller yeah, remove the IdentityFile line in first block #2019-10-03 18:03:02 ghadi Comment out the first block #2019-10-03 18:03:45 Joe Lane Hey, it works! #2019-10-03 18:05:34 Joe Lane So, while I'm glad this was fixed for me, are the github instructions conflicting with what Jsch needs? #2019-10-03 18:08:07 Joe Lane Either way, thank you both for your assistance! I see similar advice was given by Alex on the jira issue. #2019-10-03 18:10:43 dominicm The problem is, I can't figure out what would set a version resolver! There's nothing in the default service locator code. #2019-10-03 18:11:40 ghadi @lanejo01 JSch is going to be a constant source of headaches #2019-10-03 18:38:35 dominicm Okay. Figured out what sets the version resolver, but I don't understand how I'm getting java.lang.ClassCastException: Cannot cast org.apache.maven.repository.internal.DefaultVersionResolver to org.eclipse.aether.impl.VersionResolver when defaultversionresolver is like this:
public class DefaultVersionResolver
implements VersionResolver, Service
#2019-10-03 18:42:39 dominicm Doesn't that explicitly state that it's a VersionResolver?! #2019-10-03 18:44:35 seancorfield Is it crossing a class loader boundary perhaps? #2019-10-03 18:56:40 alexmiller yeah, must be same class loaded from different loader #2019-10-03 18:57:39 alexmiller looks like you're buried in some capsule stuff - doesn't that do weird things with classloaders? #2019-10-03 18:59:09 alexmiller or is this just setting up capsule stuff #2019-10-03 19:04:50 alexmiller I'm not sure what the setup is, but there must be same class available via different loaders somehow here #2019-10-03 19:05:11 dominicm This is setting up capsule stuff, so no classloaders involved here yet. #2019-10-03 19:05:32 dominicm capsule is very good on the classloader front fwiw btw. It just extracts to a directory and runs it, much like leiningen. OneJar is the classloader one. #2019-10-03 19:10:58 dominicm I'll see if I can make this a minimal repro at least. Remove pack from the equation. #2019-10-03 19:16:25 dominicm rlwrap: Oops, crashed (caught SIGABRT) - this should not have happened!If you need a core dump, re-configure with --enable-debug and rebuildResetting terminal and cleaning up...
Today is not my day apparently! 🙂 #2019-10-03 19:22:24 alexmiller wow #2019-10-03 19:22:40 ghadi that happens a lot with long lines #2019-10-03 19:23:09 ghadi probably a method to constrain it. haven't looked #2019-10-03 19:23:20 ghadi long like 10MB lines, in my experience #2019-10-03 19:29:09 andy.fingerhut I sometimes idly wonder how many different behaviors exist among line-based utility programs for very very long lines. #2019-10-03 20:51:46 dominicm What I don't understand is how simply commenting out oz would change things. How does that cause a classloader?! #2019-10-03 20:52:10 alexmiller I have no idea #2019-10-03 20:59:56 alexmiller I don't have a theory for why this is happening, just trying to work backward from the evidence #2019-10-03 21:00:42 dominicm I can't create a minimal repro either. Taking the exact pack code and placing it in another project stops it from happening... 🤔 #2019-10-03 21:00:51 alexmiller can you get pack out of it? Just call tools.deps with oz stuff? #2019-10-03 21:00:59 alexmiller I can't remember if I did that when it was logged or not #2019-10-03 21:01:44 dominicm Doing that doesn't throw. There's something contextual/environmental that I am not accounting for. #2019-10-03 21:02:36 alexmiller are you running via clj? if so, what's the classpath? Are there any uberjars such that same class has two possible sources? #2019-10-03 21:04:24 dominicm Running via clj. The classpath is pretty big because of oz. #2019-10-03 21:22:58 dominicm duplicates looking likely #2019-10-03 21:23:14 dominicm clojure -Spath -A:pack:packit | (while IFS= read -rd: line; do jar tf "${line}"; done) | sort | uniq -d I did this and got results #2019-10-03 21:38:49 dominicm I know why my repro isn't working, it's because I'm not co-locating oz and t.d.a on the classpath in my repro, but I am when I load pack into an existing project. That's also why this happens when used with add-lib. #2019-10-03 21:40:28 dominicm It's nothing to do with dependency resolution (not really anyway). It's to do with the classpath tda is running under.
I saw maven duplicates with the above classpath, so I'm guessing that's the root cause. Although I've never seen issues like this where one wins out over the other in different places. #2019-10-03 22:07:14 hiredman the classpath isn't canonical, classloaders are, the classpath is interpreted by some classloader, so if some wants a classloader that behaves sort of like the system classloader (or whatever the root classloader is called these days) it can make a classloader that interprets the classpath but has a slightly different behavior #2019-10-03 22:21:33 alexmiller Maven has all kinds of magic dependency injection and it’s entirely possible there is some service config somewhere that is giving weird behavior. Usually I just avoid using all of that stuff when I call the maven APIs but it’s surprisingly invasive. #2019-10-04 07:35:33 dominicm The alternative in this case is creating impl classes, and I'm not sure if that's better or worse. It's unclear whether that's a stable api to me, because the artefact is named impl too #2019-10-04 07:57:03 dominicm So the repro is this:
;; deps.edn
{:paths ["src"]
:deps
{org.clojure/tools.deps.alpha {:mvn/version "0.7.541"}
metasoarous/oz {:mvn/version "1.6.0-alpha5"}}}
;; src/io/dominic/tda_npe/core.clj
(ns io.dominic.tda-npe.core
(:require
[clojure.tools.deps.alpha :as tools.deps]))
(def deps-map
'{:paths ["src"]
:deps {org.clojure/clojure #:mvn{:version "1.9.0"} }
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}})
(defn -main
[& _]
(tools.deps/resolve-deps deps-map {}))
#2019-10-04 08:23:31 dominicm clojure -Spath | while IFS= read -rd: line; do
if [ -f "${line}" ]; then
jar tf "${line}" | while read -r file; do echo "${line}: ${file}"; done
fi
done | sort -k2 | uniq -f1 --all-repeated=separate
Here's a script for finding duplicates on the classpath 🙂 #2019-10-04 08:33:34 dominicm metasoarous/oz {:mvn/version "1.6.0-alpha5"
:exclusions [org.apache.maven/maven-aether-provider]}
This fixes it (haven't tested oz). Looks like there's 2 sources for the default impl of VersionResolver! #2019-10-04 08:34:43 dominicm It's coming transitively from pomegranate, so I think you can't have pomegranate and t.d.a on the classpath together without this exception. #2019-10-04 12:02:15 vlaaad This is amazing, thank you very much! #2019-10-04 12:42:03 delaguardo you are welcome to open an issue if something isn’t working as expected) #2019-10-04 18:34:34 seancorfield Adding this to next.jdbc to matrix test on Java 8 and Java 11. Thank you! #2019-10-04 18:52:52 seancorfield Wow, it takes a long time for those jobs to get started! 21 minutes for the first one. Still waiting for the second one in the matrix. #2019-10-04 19:04:23 seancorfield 29 minutes end-to-end. It'll be interesting to see how they tune that during the beta. CircleCI started almost immediately (and completed in 30 seconds, compared to an actual run time of about a minute on GitHub). #2019-10-04 19:36:09 vlaaad For me it usually starts immediately... #2019-10-04 19:37:43 seancorfield Maybe it was just because this was the first time I've used it? I'll report back on subsequent runs. #2019-10-04 20:54:54 seancorfield Just pushed another commit and, yeah, it started up nearly immediately. #2019-10-05 10:56:36 ikitommi hi. I have a transitive deps issue. metosin/malli depends on both borkdude/sci and borkdude/edamame via sha. sci uses edamame via real mvn version. When I try to use malli, the versions conflict:
➜ ~ clj -Sdeps '{:deps {metosin/malli {:git/url "" :sha "28b4ee63bd571ce612e8cd0e9dab6ef86722597e"}}}'
Checking out: at 28b4ee63bd571ce612e8cd0e9dab6ef86722597e
Downloading: borkdude/edamame/0.0.6/edamame-0.0.6.pom from
Error building classpath. Unable to compare versions for borkdude/edamame: {:mvn/version "0.0.6", :deps/manifest :mvn} and {:git/url "", :sha "9b3e825b65ec6fe0dc536988fd036e6e681813de", :deps/manifest :deps, :deps/root "/Users/tommi/.gitlibs/libs/borkdude/edamame/9b3e825b65ec6fe0dc536988fd036e6e681813de"}
#2019-10-05 10:58:14 ikitommi If I add explicit sha to edamame from command like, it works, but is not what I would like to be needed. #2019-10-05 11:00:40 ikitommi e.g. this works:
➜ ~ clj -Sdeps '{:deps {metosin/malli {:git/url "" :sha "28b4ee63bd571ce612e8cd0e9dab6ef86722597e"}, borkdude/edamame {:git/url "", :sha "9b3e825b65ec6fe0dc536988fd036e6e681813de"}}}'
#2019-10-05 11:21:22 ikitommi quick-hacked this so that malli depends on edamame mvn version. Still, curious could it be made to work with just Sha dep to it #2019-10-05 11:27:00 dominicm What do you think should happen here? The problem is that maven and git don't have a comparison mechanism that makes sense. #2019-10-05 11:34:50 borkdude @ikitommi I think it's best to depend on a mvn version of edamame then from now on. I'll try to do that with sci as well. #2019-10-05 12:01:32 alexmiller This can be made to work, but I haven’t done so yet. Maven artifacts generally record their sha at build time so it is possible to version compare them. #2019-10-05 12:02:43 borkdude what if you have two branches that are diverged from the same maven version? #2019-10-05 13:58:01 alexmiller This is just back to the same rules we use for git comparisons - if one version is a descendant of another, its preferred. If there is no ancestor relationship, they aren’t comparable #2019-10-05 13:59:28 alexmiller In that case, it will throw and you need to be explicit at the top level about which dep to use (top dep version always used) #2019-10-07 15:52:14 alexmiller if anyone here is an add-lib user, can I ask the scenario where you're using it? is it
1) to definitely add a lib you know you need from prior experience
2) to add a lib you might want to use but are not sure if you need
3) to do a task at the repl that you definitely don't want in your deps.edn #2019-10-07 17:16:03 seancorfield Is 3) meant to include situations where you have a "scratch" REPL running just to try stuff out (e.g., helping beginners on Slack) and it would be less convenient to restart that REPL with difference deps rather than just adding a lib to the REPL? For me, such situations are usually not accompanied by deps.edn at all. #2019-10-07 17:18:04 alexmiller you always implicitly have a deps.edn, even if it's just including clojure from the system deps.edn #2019-10-07 17:19:26 alexmiller sounds like a mixture of 2 and 3 #2019-10-07 17:19:26 seancorfield Well, yeah, I meant the "project" (current directory) one. Libraries that I use add-lib for wouldn't generally go in either of the other files. #2019-10-07 17:20:54 seancorfield I have a scratch directory that has deps.edn with {} in it and I usually start a plain REPL with -Sdeps on the command-line, but if add-lib was easier to use, I'd just start that plain REPL and then add-lib in anything I wanted to experiment with and just leave it running (instead of restarting with different deps). #2019-10-08 07:47:43 plexus For me it would be mostly 1), especially in the early stages of a project I want to be able to add dependencies without having to restart the process. It could also be 2) if it's a lib I haven't used before, to be able to quickly evaluate it before committing to it. #2019-10-07 15:56:43 dominicm I have played around with the idea of 3, maybe even for use in dev tooling libraries. But I haven't actually done it yet. #2019-10-07 15:58:35 alexmiller by which you mean you would have a lib that called add-lib? #2019-10-07 16:00:27 dominicm Yeah. Although essentially it's a library of things you'd do at the repl.
I can't remember the details of why I was doing this now. Possibly to keep things "light" at someone's request, or maybe because I wanted to load things for dev time based on the config.edn #2019-10-07 15:57:03 alexmiller and if you answered 1 or 2, how do ensure the dep gets added to deps.edn? do you do it at the same time you add-lib or rely on remembering later? #2019-10-07 15:58:29 delaguardo tests #2019-10-07 15:59:06 alexmiller not sure what that means? you add tests and rely on that failing later to tell you to add the lib? #2019-10-07 16:01:33 delaguardo yes, not a perfect solution, but covers most of the problems
something like Could not locate chesire/core__init.class, chesire/core.clj or chesire/core.cljc on classpath. usually is enough to remember what went missing #2019-10-07 16:11:16 kenny Yep, pretty much the same thing as @U04V4KLKC. If the lib ends up getting used in the project, it will get added to a namespace's :requires. This will cause the project to fail CI tests if someone forgot to add it to the deps.edn. #2019-10-07 16:40:54 dharrigan remember later #2019-10-07 17:14:23 seancorfield For 1) I do it at the same time because it's a deliberate act. For 2) I do it later, once I've decided whether the lib is really what I need (so, yeah, I run the risk of forgetting but if I really need the lib, it'll end up required in the code base somewhere so tests will fail if I have forgotten to add it). #2019-10-07 15:58:43 dominicm Later. Although I've been working on something which would remember what is add-lib'd and allow me to commit it. And also an alternative entry point for loading it and committing simultaneously. #2019-10-07 15:59:42 alexmiller I'm currently exploring similar ideas #2019-10-07 16:01:55 dominicm As an fyi, I currently don't find the git tag rewriting useful.
I put comments into my deps.edn, and losing those is annoying. I've got a few toys that use rewrite-clj to avoid that (including a implementation of the git tag stuff), but they're difficult to write. #2019-10-07 16:03:02 dominicm Or rather, it's useful, but that side effect makes it problematic. #2019-10-07 16:03:16 alexmiller yeah, I'm probably going to remove that #2019-10-07 16:03:52 dominicm I know :) but the external version would have the same problem, so would a commit-add-lib! #2019-10-07 16:04:11 alexmiller well, I'm more exploring a "refresh" type option #2019-10-07 16:04:24 alexmiller so you would be the one doing the add #2019-10-07 16:04:56 dominicm Ah sync-deps, I think someone tried that here. They requested knowledge of the aliases in use so they could merge them together :) #2019-10-07 16:08:53 alexmiller "sync" is ambiguous about the direction of the sync. "refresh" is deps.edn -> repl #2019-10-07 16:13:37 dominicm That's fair. Refresh made me think of tools namespace and I struggled to separate the two. #2019-10-07 17:16:52 alexmiller How do you know the version of the lib for add-lib? Do you:
1) Look it up on github or other source
2) Use LATEST or RELEASE #2019-10-07 17:17:33 seancorfield I use "RELEASE" a lot when I'm just experimenting with a library. I only bake the version into deps.edn once I know I'm going to use it. #2019-10-07 17:18:29 seancorfield The "refresh" scenario wouldn't cover a "scratch" situation where you're just trying a lib out in the REPL and don't want deps.edn created/updated. #2019-10-07 17:23:45 alexmiller yes, I'm aware. investigating alternatives, making tables :) #2019-10-07 17:20:12 dharrigan Mostly 1, sometimes 2. #2019-10-07 17:28:04 dominicm I usually look it up, I need to look up the coord anyway. I don't memorize those. #2019-10-08 06:12:26 orestis Just a data point - sometimes I’m repl’ed in a remote environment and I would find it useful to add a library without having to touch the remote deps.edn file, which might be tucked in a jar in any case. It’s on me to remember to do it after the fact, and that would require a proper deploy step anyhow. #2019-10-08 06:14:38 orestis I’m sure the table Alex is building will be much more nuanced, but the obvious (to me) building block is a function that can add a fully specified dep to the runtime. Figuring out the latest version, refresh the env from a deps.edn could be userland concerns, no? #2019-10-08 13:19:45 dominicm This is what we have, so I think these might be baked in as well #2019-10-08 21:49:12 Michaël Salihi Hi ! I I would like to know if it's possible to populate deps.edn dependencies in command line or some externals tools ?
Right now, I use clj-refactor with cljr-add-project-dependency in Emacs but it only works when a REPL is already running. #2019-10-08 22:23:01 ghadi @admin055 it’s just data - you can programmatically generate it #2019-10-09 01:43:45 mpisanko hi there, pardon if totally missed this (but google didn't give me any answers). my question is: is there an alternative to lein's pedantic? in tools.deps #2019-10-09 02:14:45 seancorfield @mpisanko No. It's also worth bearing in mind that the algorithm for selecting versions is different in t.d.a. so there should be fewer potential conflicts. Or at least, fewer counterintuitive version selections 🙂 #2019-10-09 02:16:25 seancorfield I think there's still an undocumented :verbose true option you can add to deps.edn so that t.d.a. explains all of its decisions... #2019-10-09 02:17:35 mpisanko right, that's the premise of improved dependency resolution i guess. thanks @seancorfield! #2019-10-09 02:42:51 seancorfield Looks like :verbose needs to be under an alias, not at the top-level... {:aliases {:v {:verbose true}} ,,,} and then clojure -A:v will show the resolution output as t.d.a. works. #2019-10-09 02:57:05 alexmiller actually there are tools that can help you with this in clj #2019-10-09 02:57:36 alexmiller in particular, check out depot or deps-ancient #2019-10-09 03:01:27 seancorfield Those will detect old (outdated) deps in your project -- lein pedantic detects possible version conflicts (usually in transitive deps) and warns about them (and prevents command execution if any potential conflicts are present, as I recall) so it's not really equivalent to anything on that list...? #2019-10-09 03:02:31 seancorfield I think t.d.a.'s approach makes much more sense and you can either eyeball things with -Stree to see what actually got picked, but you can only see the potential conflicts if you ask t.d.a. to be verbose, as far as I can tell. #2019-10-09 03:03:24 seancorfield (we use depot at work, and it's very useful for flagging deps that could be updated -- but does not show potential conflicts) #2019-10-09 03:10:39 seancorfield Looking in more detail at some verbose output from t.d.a. I can see its decision process cuts off some paths that I think Leiningen considers... I'd have to try to repro in lein to be sure... (this has piqued my curiosity!). #2019-10-09 03:20:02 seancorfield Yes, confirmed. Example: deps.edn specifies Clojure 1.9.0 and seancorfield/next.jdbc 1.0.8. t.d.a. pins those versions because they are top-level and when it sees that next.jdbc depends on Clojure 1.10.1 it just discards that due to 1.9.0 being the top-level dep. Consequently, t.d.a. goes on to select spec.alpha and core.specs.alpha based on Clojure 1.9.0. When you put the same deps in project.clj and :pedantic? :abort and ask for lein deps :tree, Leiningen also shows the transitive dependencies of Clojure 1.10.1 and warns they conflict with Clojure 1.9.0's deps, suggesting exclusions (on next.jdbc) to exclude all three of clojure, spec.alpha, and core.specs.alpha (and then it aborts). #2019-10-09 03:21:15 seancorfield (if you tell lein to exclude org.clojure/clojure that is enough to satisfy it) #2019-10-09 07:48:39 dottedmag @alexmiller Please have a look at TDEPS-56 when you have time. I have posted a patch and an upsides/downsides table some time ago, and there was no further discussion on the topic. #2019-10-09 07:59:57 Michaël Salihi @ghadi Yes, I wanted to know if there was already existing tool to manage dependencies before to begin development.
I find this tool who do the job https://github.com/hagmonk/find-deps #2019-10-09 20:30:17 jrwdunham Is it possible to use an alias to override a github library with a local root library? #2019-10-09 20:32:53 jrwdunham {:deps {github-myorg/myrepo {:git/url "some-url" :sha "some-sha"}}
:aliases {:local {:override-deps {myrepo {:local/root "/abs/path/to/my/local/repo/"}}}}} #2019-10-09 20:33:14 jrwdunham ^ doesn't seem to do what I want. #2019-10-09 20:33:37 jrwdunham presumably because the lib names github-myorg/myrepo and myrepo differ #2019-10-09 20:36:52 jrwdunham d'oh. It looks like I can just stop using the symbol github-myorg/myrepo for the GH dep and instead use myrepo as with the local dep. I didn't realize that was just a naming convention and not a requirement. #2019-10-09 20:42:15 dominicm Extra-deps instead of override #2019-10-09 20:42:52 jrwdunham but wouldn't that result in two deps with the same namespaces? #2019-10-09 20:43:38 jrwdunham to be clear, this now seems to do what I want: #2019-10-09 20:43:52 jrwdunham {:deps {myrepo {:git/url "some-url" :sha "some-sha"}}
:aliases {:local {:override-deps {myrepo {:local/root "/abs/path/to/my/local/repo/"}}}}} #2019-10-09 20:48:53 seancorfield Yup, myrepo in the above is the coordinate for your dependency: the group and artifact ID, as if it were myrepo/myrepo -- and the coordinate has to match since that's what you are overriding. #2019-10-09 20:49:27 seancorfield (well, technically, the "coordinate" is the group/artifact and the version information but...) #2019-10-09 20:50:16 seancorfield So it isn't "just a naming convention" -- it has a specific meaning. #2019-10-09 20:57:31 jrwdunham Thanks @seancorfield that clarification is helpful. I should have been more clear: I intended to write that I was under the false assumption that if a coordinate used the :git/url key in its version info that it had to have a group containing the github- prefix. #2019-10-09 21:00:09 seancorfield If you're not trying to override a "regular" dep with a git dep, then you can give it any unique name you want, but using your github organization (username) and the repo name means it's not likely to conflict with a "regular" dep. #2019-10-09 21:03:39 seancorfield If you're working with a regular dep (group/artifact) with a Maven version, you can just swap out the version part for a :git/url part or :local/root part and keep the same group/artifact to effectively override the Maven version with that git or local version -- and that will work even if it is a transitive dependency of something else you're depending on (since you'd have it as a top-level dep). #2019-10-09 21:03:51 seancorfield Does that help, or just make it more confusing @jrwdunham? #2019-10-09 21:15:40 jrwdunham @seancorfield I am left a bit more helped and a bit more confused. So, in your example, since you are not declaring a top-level :deps for org.clojure/tools.deps.alpha, you use :extra-deps in your :deps alias to effectively override the Maven dependency that would be installed otherwise. Is that accurate? #2019-10-09 21:52:52 seancorfield I use :extra-deps there to ensure it is added (with that git version). :override-deps only takes effect if something else is bringing in that dependency. #2019-10-09 21:54:44 seancorfield (in this case, it becomes a top-level dep so even if something else brings in t.d.a. as a transitive dep, my selection takes precedence) #2019-10-09 23:24:19 cfleming Has anyone had problems using s3 repos with deps under Java 11? #2019-10-09 23:25:09 cfleming Some Cursive users are having problems which is reported as not being able to access java/sql/Timestamp, when using lein that’s usually resolved by upgrading the s3 wagon. But it looks like the deps version is baked in. #2019-10-09 23:26:24 cfleming Is anything required to be added to the module classpath, or anything like that? #2019-10-09 23:30:09 alexmiller no, lots of people doing this, haven't heard any issues #2019-10-09 23:30:24 alexmiller java/sql/Timestamp is in particular not in the base module #2019-10-09 23:30:34 cfleming Ok, I’ll see if I can figure it out, and I’ll report back if I do. #2019-10-09 23:30:59 cfleming Right, that’s what’s been giving problems since IntelliJ started using JBR11 as its base JRE. #2019-10-09 23:31:01 alexmiller so lein's trick of adding itself to the bootclasspath doesn't work #2019-10-09 23:31:20 alexmiller as at boot time, only the base module is loaded #2019-10-09 23:31:42 cfleming I’m pretty sure it’s not a bootclasspath issue, since this is executed within Cursive and I don’t use it. #2019-10-09 23:31:56 cfleming Well, IntelliJ doesn’t use it, more accurately. #2019-10-09 23:32:03 alexmiller yeah, didn't make sense to me either #2019-10-09 23:32:09 cfleming I run lein in an isolated classloader using shimdandy. #2019-10-09 23:32:10 alexmiller but it matches those symptoms #2019-10-09 23:32:32 alexmiller what do you put in the isolated classloader? and where is it rooted? #2019-10-09 23:32:57 alexmiller if it's rooted off the base loader or app loader or whatever they call it, it might only have the base module available #2019-10-09 23:35:30 cfleming Yes, I think that’s probably the case. It’s rooted off the parent classloader for a boostrap class I use, which IIRC is the app classloader. It almost certainly won’t have the SQL module available. #2019-10-09 23:36:02 alexmiller in java 9+ that loader is represented as null (the bootstrap loader). it only has access to the base module #2019-10-09 23:36:18 cfleming But my impression was that some of the older s3 wagons had problems with this that were resolved in later versions - certainly using lein, upgrading the wagon usually fixes the issue. #2019-10-09 23:36:40 alexmiller deps uses a newer version of the s3 wagon than the lein plugin does #2019-10-09 23:36:52 cfleming But my understanding of how all this works in the brave new world is sadly incomplete. Looks like I’m going to have to figure it out. #2019-10-09 23:37:02 cfleming Hmm, interesting. #2019-10-09 23:37:05 alexmiller well, maybe it's been updated since I last looked, not sure #2019-10-09 23:37:14 cfleming Ok, I’ll see if I can reproduce the issue and debug it. #2019-10-11 14:25:37 rapskalian #2019-10-11 14:28:09 rapskalian If my lib makes use of a dep, but I fully expect the users of the lib to also make use of that dep, it made sense logically to not include those deps as transitive dependencies. But I lack a full understanding of how t.d. resolves conflicts… #2019-10-11 14:28:24 rapskalian I probably just need to RTFM 🙂 #2019-10-11 14:32:45 alexmiller this is certainly under-documented #2019-10-11 14:33:23 alexmiller In general, I think it makes sense for a lib to include all the dependent libs it needs in its explicit dependencies #2019-10-11 14:34:00 alexmiller the way tda resolves conflicts is generally to pick the newest version #2019-10-11 14:34:38 alexmiller with the caveat that it picks whole subtrees and in some cases that upstream libs higher in the tree may dominate #2019-10-11 14:34:54 alexmiller however, one big hammer is that the versions at the very top (presumably the app) always win #2019-10-11 14:35:10 alexmiller so the end user always has final say #2019-10-11 14:36:03 alexmiller so if the app includes both your lib and the datomic cloud-client lib, then it's explicitly naming the datomic lib and that version will be used, which is what you want here #2019-10-11 14:36:57 alexmiller let me know if that doesn't make sense #2019-10-11 14:37:14 rapskalian Okay, that makes sense, thanks. Explicit deps are certainly much simpler as well. It’s nice to be able to lean on t.d.a. a bit there. #2019-10-11 14:38:38 rapskalian I’m going to remove my :provided alias for now and just go with the simplest solution. Thanks, Alex. #2019-10-11 14:57:27 dominicm The only thing close to provided deps would be optional specs. #2019-10-11 17:01:13 rapskalian @dominicm is there a t.d. way to do that? Best I’ve seen people do is add a conditional flag surrounding calls to s/def… #2019-10-11 17:02:04 dominicm No, nothing special. Just that or optional namespaces. #2019-10-11 17:36:33 alexmiller t.d way to do what? #2019-10-11 20:37:03 rapskalian @alexmiller to optionally also bring in the specs…I’m not sure exactly what that would mean honestly. Potentially some way to specify a dependency along with additional aliases, like:
:deps {me/sweet-dep {:mvn/version "XYZ" :aliases [:specs]}}
but this is potentially not practical, or even desirable. #2019-10-11 20:41:38 seancorfield @cjsauer clojure.java.jdbc and next.jdbc have "optional specs" -- they have them in a separate namespace in the project and if you want to use them, you just require the *.specs namespace. That allows the projects to run on Clojure versions older than 1.9 (i.e., before spec was created) but the specs are still bundled with the project/artifact. #2019-10-11 20:42:46 alexmiller ^^ #2019-10-11 20:42:56 rapskalian Yeah that seems much cleaner #2019-10-11 20:43:02 alexmiller or put them in separate libs, but that's a lot more work #2019-10-11 20:43:15 seancorfield (well, next.jdbc requires at least Clojure 1.10 because it depends on clojure.datafy but I could probably make that feature optional as well) #2019-10-11 20:44:58 rapskalian My time in web dev forces this desire to keep the dep small in size…but for jars that really doesn’t matter I suppose…
For example, in this datomic ions lib I’m working on, I have an example project included in a separate source folder. I toyed with ideas for not including it in the final artifact, but there’s really no gain from that effort (and it would even prevent me from deploying the example, because ions uses your top-level deps, i.e. it doesn’t take aliases into account) #2019-10-11 20:45:53 rapskalian Seems fine to have examples bundled inside of a lib, yeah? #2019-10-11 20:47:00 rapskalian Not to mention that the cljs compiler will strip any unused code out anyway. My brain is still catching up to these awesome tools 😅 #2019-10-11 20:50:15 seancorfield I'd probably create a separate project for the example stuff that depended on your main project and then just document it copiously in the main project. But, as you say, source is tiny so shipping an example in the main project doesn't hurt as long as it's all in separate namespaces that users can ignore if they aren't interested in it. #2019-10-11 20:51:37 seancorfield Also, you can have a git dependency that uses a subfolder of the repo -- which might be another was to offer it as an example to folks? #2019-10-11 20:52:33 seancorfield :deps/root #2019-10-11 20:52:36 rapskalian It actually was a separate repository, but it made developing the lib such a pain because changes are happening rapidly at the moment. It’s likely something that can be extracted once the API stabilizes tho. #2019-10-11 20:52:59 rapskalian Oh cool. Is that different than :local/root? #2019-10-11 20:53:27 seancorfield Yes, it goes with :git/url to tell t.d.a. to navigate into the specified location within the repo. #2019-10-11 20:54:25 rapskalian Interesting…I might have to try that. What is the REPL experience like with this? It’d be great to be able to eval things and have the example pick them up instantly. #2019-10-11 20:55:14 seancorfield "It’d be great to be able to eval things and have the example pick them up instantly." -- that is what I would expect with an RDD workflow. #2019-10-11 20:57:19 seancorfield When I'm working on a project, I do everything via the REPL. So I either start the apps/services from inside the REPL, or I add the JVM options to start a Socket REPL when the app/service starts up (such as on QA/production). Then I connect my editor to that REPL and eval code as I go, so changes are picked up immediately by the running apps/services. #2019-10-11 20:58:07 seancorfield We sometimes apply hot fixes to production processes that way -- zero downtime 🙂 #2019-10-11 20:59:03 rapskalian Bold! But let’s say one of your services depends on either a :local/root or deps/root dependency, and you realize you need to make a change in that dep. Can you do that easily without rebooting the REPL? #2019-10-11 20:59:36 rapskalian In Cider I can jump into a lib that isn’t mine part of the current project, and make changes and eval them, and it works, but obviously those aren’t “saved” in the traditional sense #2019-10-11 21:01:28 rapskalian I’m sure this becomes much less of an issue once the lib begins to stabilize. At the moment, my example serves as something of a test bed, so changes between the two are happening constantly. The convenience of having both in the same deps project is really nice, but it’d become less useful as the lib changes slowed. #2019-10-11 21:03:15 rapskalian When they were in separate repos, I found I was having to reboot my REPLs often. There might be a better way tho. Especially with :local/root, which I admittedly haven’t experimented with much. #2019-10-11 21:07:46 seancorfield I'm not quite sure what you're asking. If you evaluate code into the REPL, the changes take effect immediately. So, yeah, if you have the source of the lib you're depending on, you can edit it and evaluate it into your REPL without needing to restart it. #2019-10-11 21:10:47 seancorfield When you're developing stuff locally and you're code is still volatile, you can use :local/root to point to the folder the source is in and just edit/eval it into the REPL you're already using for your example project. #2019-10-11 21:14:39 rapskalian Maybe this is a limitation of cider. REPLs in cider seem to be “scoped” to the current project. If I open the source file of another folder, I no longer seem to have a REPL connected 🤔 #2019-10-11 21:25:05 dominicm You're looking for the sesman related commands #2019-10-11 21:27:25 rapskalian 🙏 #2019-10-11 21:27:46 rapskalian Ahhh there we go! It let me link the file to an existing REPL session. #2019-10-11 21:28:02 rapskalian this changes everything 😭 #2019-10-11 21:29:48 rapskalian @U04V70XH6 suddenly I see why my questions make no sense…all this time everyone else has been eval’ing arbitrary source code. #2019-10-11 21:35:07 seancorfield I assume that's to do with CIDER's ability to work with several open REPLs, one for each "project"? I remember when I switched from Emacs/CIDER/nREPL to Atom/ProtoREPL (three years ago) at first I was annoyed that ProtoREPL didn't let me just switch between multiple REPLs but over time I just adopted a workflow where I can work in one REPL across multiple projects -- and now I tend to run a single "everything" REPL for days or even weeks, and just eval in any source code I need to work with. #2019-10-11 21:37:06 seancorfield (and I'm now figuring out how best to work with add-lib on the t.d.a. branch so I don't even need to restart a REPL to add new library dependencies) #2019-10-11 21:46:52 rapskalian CIDER actually had that functionality way back when, I really miss it. Hot-loading deps was super convenient. #2019-10-11 22:05:12 seancorfield I have a :deps alias with the add-lib branch and then (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
and then (add-lib 'some/new-lib {:mvn/version "RELEASE"}) and it downloads the new library and then it can be required. #2019-10-11 22:06:05 seancorfield What I was missing was two lines of code to set up a DynamicClassLoader immediately before starting my Socket REPL and REBL (I have a small ws.dev.repl/-main function for doing that, based on what it finds on the classpath). #2019-10-11 22:07:19 seancorfield (I've already opened an issue against REBL to suggest that it starts its REPL in a DCL) #2019-10-12 12:57:19 dominicm add-lib works with cider too fwiw. I added support a little while ago. #2019-10-11 21:15:44 seancorfield Hmm, I don't use CIDER. #2019-10-11 21:15:46 rapskalian The chances that I’m doing something wrong are overwhelmingly high tho 😅 #2019-10-11 21:16:13 seancorfield I use Atom with Chlorine so I can use simple Socket REPLs with no additional dependencies in my code. #2019-10-11 21:17:23 seancorfield So, for me, there is "only one REPL" and I can evaluate any and all code into it. I like simple tooling. CIDER is not simple. nREPL is not simple. #2019-10-11 21:18:57 seancorfield Stu Halloway was interviewed on the ClojureScript podcast about his REPL usage and he uses really minimal tooling (a plain REPL with inferior mode in Emacs I think he said). I think Rich is the same. #2019-10-11 21:20:14 seancorfield Chlorine gives me auto-complete using just the built-in features in Clojure, which I also like (it used to require Compliment for that -- which is also part of CIDER/nREPL). #2019-10-11 21:20:51 seancorfield It's also why I was so happy to switch from lein/`boot` to the new CLI/`deps.edn` stuff. Simple tooling. #2019-10-11 21:26:18 rapskalian Hm, I may have to give inf-clojure a try at some point. Plain old eval would likely be just fine honestly. And I had a similar experience moving from lein to deps.edn…breath of fresh air. #2019-10-15 14:48:07 murtaza52 how do I clear my cache, and force compilation with clj, line what lein clean used to do ? #2019-10-15 14:50:40 dominicm clj -Sforce #2019-10-15 14:50:50 dominicm Or rm .cpcache #2019-10-15 14:55:30 murtaza52 .cpcache contains compiled class files ? #2019-10-15 15:02:01 andy.fingerhut I believe .cpcache does not contain any compiled class files, but in some cases it will contains names of JAR files, e.g. in your $HOME/.m2 directory. #2019-10-15 15:03:02 andy.fingerhut clj commands do not normally save any results from compilation, unless you have some alias that causes evaluation of expressions that write .class files (again, I am pretty sure there, but others will correct if I am missing something). #2019-10-15 15:06:57 seancorfield @murtaza52 unless you explicitly AOT code, clj does not produce .class files at all. #2019-10-15 15:28:32 murtaza52 thanks guys #2019-10-16 20:13:30 unbalanced eya... do we have anything like clj -A:new templates yet? #2019-10-16 20:50:21 seancorfield @goomba clj -A:new can use all of the Leiningen and Boot templates (they'll just produce whatever they are designed to produce). #2019-10-16 20:50:29 unbalanced whaaaaaaaaaaaaaaaaaaaaaat #2019-10-16 20:50:46 seancorfield But I doubt there are any clj-specific templates out there (yet). #2019-10-16 20:51:03 unbalanced psssshhhh I made one #2019-10-16 20:51:54 unbalanced you made it easy enough even for a @goomba ™ to figure out #2019-10-16 20:53:32 unbalanced cool! 😄 (man I was dangerously close to getting "bing"ed again hahaha) #2019-10-16 20:54:08 seancorfield The main issue is that clj -A:new can pull in templates from git or local deps (which lein/boot cannot) and those would never be discovered by #2019-10-16 20:56:34 unbalanced cough this one appears to be missing a vote button 😅 #2019-10-23 09:55:06 practicalli Give the issue a thumbs up emoji, that votes for it #2019-10-16 21:16:12 cfleming Is it new that -Sdescribe shows deps.edn at the end of :config-files, and not just the system and user deps? #2019-10-16 21:28:50 alexmiller I think it's always been that way #2019-10-16 21:29:40 alexmiller I don't remember changing anything about it at least #2019-10-16 21:31:44 seancorfield @cfleming It only shows that if you're in a folder that has deps.edn in it. #2019-10-16 21:31:50 cfleming Yeah, looking at the history of the script looks like it’s always been like that. #2019-10-16 21:32:20 cfleming @seancorfield Oh, I see, you’re right. #2019-10-16 21:33:42 cfleming @alexmiller that’s the source of the weird bug the other day where stijn was getting a spurious dep - he’s on linux, which starts IntelliJ in his home directory, and he had half a deps.edn file in his home dir. #2019-10-16 21:33:56 alexmiller ah, cool #2019-10-16 21:34:23 alexmiller and by cool, I mean very confusing and not cool #2019-10-16 21:34:27 cfleming I think that I usually run -Sdescribe without setting the CWD since I didn’t realise it was significant. #2019-10-16 21:34:42 alexmiller hey, it is #2019-10-16 21:34:42 cfleming It’s a weird one, for sure. #2019-10-16 21:34:49 cfleming Thanks 🙂 #2019-10-16 21:35:21 cfleming Any chance of ever getting that vector replaced by a map with actual names for the different paths? #2019-10-16 21:36:07 cfleming It’s really ambiguous now - this has also caused confusion around the system file being bundled in t.d.a, too #2019-10-16 21:36:54 cfleming I have a feeling I’ve asked for that previously, but I can’t remember the conclusion. #2019-10-16 21:50:43 seancorfield It needs to be ordered -- that's how the files are merged -- and, for a while, the system one was removed from the list because it got built in (but I think it was restored, at least until tooling catches up with that change?). #2019-10-16 21:50:55 seancorfield Well, it was restored, but I think that's why 🙂 #2019-10-16 21:52:11 seancorfield Lots of tooling depends on the vector / order so it couldn't change to be a map, but I guess one or more new keys could be added to the overall describe result. #2019-10-16 22:07:44 alexmiller "Lots" == ? #2019-10-16 22:08:10 cfleming Cursive, for one #2019-10-16 22:08:27 alexmiller well that's the one I was aware of :) #2019-10-16 22:08:44 alexmiller I was wondering what else was in the "lots" #2019-10-16 22:09:52 cfleming Cursive doesn’t really depend on the order per se, but does want to understand which of the files are system/user config files and which are the project files. #2019-10-16 22:10:23 alexmiller that should be easier #2019-10-16 22:12:13 seancorfield True, since the API changed, there should be fewer-to-none using the describe output now 🙂 #2019-10-16 22:13:18 seancorfield I've just seen several tool authors talk about reading that list to get at the raw deps.edn files instead -- although I keep telling them not to do that! #2019-10-16 22:14:12 seancorfield depot just went through a major rewrite to "unify" its read/check behavior with its write behavior (it will update deps.edn to use the latest versions of artifacts). #2019-10-16 22:15:18 seancorfield Prior to the rewrite, the read/check behavior used the built-in t.d.a. logic and looked at the resulting deps/versions -- but the write behavior read deps.edn files directly. Now it does it on both paths which completely broke our usage of the tool 😞 #2019-10-16 22:17:55 alexmiller I sent PRs to all the projects I could find using clj -Sdescribe output this summer and it actually emits a big ugly warning to stderr if you use it, which I have seen no one mention #2019-10-16 22:19:04 alexmiller this is via the ctda.reader/clojure-env #2019-10-16 22:21:17 seancorfield I think depot just reads the local deps.edn by default or the list of files you provide on the command-line otherwise. I don't think it even respects CLJ_CONFIG. #2019-10-16 22:02:44 alexmiller I am actually working on a bunch of changes in this area right now #2019-10-16 22:04:29 alexmiller we are naming the different deps files #2019-10-16 22:04:49 alexmiller how that affects -Sdescribe output tbd (but not planning to break the existing) #2019-10-16 22:05:58 alexmiller we may need to have a conversation @cfleming to make sure you stay in sync, but I need to work through it a little more first #2019-10-16 22:07:54 cfleming @seancorfield Right, but you can still apply an ordering to the map elements (i.e. :system comes before :user comes before :project) #2019-10-16 22:08:06 alexmiller yes #2019-10-16 22:09:33 alexmiller the ordering can be independent (or more likely, fixed) #2019-10-16 22:10:30 cfleming Anyway, I’m glad to hear changes are coming, much appreciated! #2019-10-16 22:11:17 alexmiller do you call anything in clojure.tools.deps.alpha.script.make-classpath ? #2019-10-16 22:11:52 cfleming I don’t think so, let me check… #2019-10-16 22:13:12 cfleming No, I use c.t.d.a, c.t.d.a.reader and c.t.d.a.util.dir #2019-10-16 22:13:30 alexmiller cool #2019-10-16 22:14:42 alexmiller that should all be stable (well, I added ctda.reader/user-deps-location if that's useful) #2019-10-16 22:15:45 alexmiller but I am adding some features that are changing the glue between the clojure script and tda (really all logic in make-classpath) #2019-10-16 22:16:25 alexmiller so if you're replicating that logic, it will need to change #2019-10-16 22:16:43 alexmiller but I should really go finish all that so I can get it in rather than talking here :) #2019-10-16 22:19:49 cfleming Interesting, thanks. I don’t think that’s relevant to Cursive since IntelliJ doesn’t reliably receive env vars from the shell environment, but I do need to allow users to specify where their user deps.edn file is. I already allow that for users using t.d.a directly, but not for CLI users at the moment. #2019-10-16 22:20:34 cfleming Speaking of which, what’s the state of play of CLI on Windows? I haven’t followed that. #2019-10-16 22:21:04 alexmiller when you use t.d.a directly, there may be something additional you need to do #2019-10-16 22:21:57 alexmiller I wouldn't say it's done :) #2019-10-16 22:22:43 cfleming Hehe #2019-10-16 22:22:58 cfleming What is the additional thing I may need to do when using t.d.a? #2019-10-16 22:23:14 seancorfield FYI, all I use in clj-new is [clojure.tools.deps.alpha :as deps]
[clojure.tools.deps.alpha.reader :refer [default-deps
read-deps]]
[clojure.tools.deps.alpha.util.session :as session]
So I assume I'm safe @alexmiller? #2019-10-16 22:23:22 alexmiller let me finish what I'm working on and then I can have things to point at #2019-10-16 22:23:33 seancorfield Haha... OK. #2019-10-16 22:23:57 alexmiller I can't imagine clj-new is going to be affected by any of this #2019-10-16 22:24:05 cfleming Ok, I shall await it with bated breath… #2019-10-16 22:24:19 alexmiller actually it will be able to make use of some new stuff #2019-10-16 22:24:28 alexmiller but I'm going to go work on it :) #2019-10-17 14:17:39 holyjak REBL advice needed (is the a better channel?) : I run a remote nrepl server and connect to it with a local nrepl client (mostly via Cursive). I want to include REBL but run it on the local machine, not the server. Is it possible? (I know there is a nrepl middleware for REBL but I guess it assumes REBL is running on the same jvm as the repl itself). Thank you! #2019-10-17 19:49:01 Aleksander #rebl #2019-10-18 08:21:40 rickmoynihan It’s not possible. REBL has to exist in the same JVM as the forms it’s browsing. I spoke to Rich about this a while back, and IIRC it’s because to do it you need to implement weak-references over the wire; to enable GC etc. (He implemented such a system years ago in common lisp http://foil.sourceforge.net/ and I guess wanted to avoid the complexity for only marginal use-cases) #2019-10-18 09:27:43 holyjak I see, thank you! 😢 #2019-10-17 14:20:40 vlaaad there is #rebl #2019-10-17 14:23:20 vlaaad I played with rebl, can't tell anything about nrepl, but rebl uses prepl, and you can pass your own prepl function to it #2019-10-17 14:24:54 vlaaad cognitect.rebl/ui accepts :proc keyword argument, which should be a function of 2 args: in-reader and out-fn, as clojure.core.server/prepl #2019-10-17 14:25:48 vlaaad there is clore.core.server/remote-prepl, which can be transformed to satisfy :proc #2019-10-17 14:26:53 vlaaad I played with it, here is what worked for me:
(cognitect.rebl/ui :proc
#(clojure.core.server/remote-prepl
"localhost" 50505
%1 %2
:valf (partial clojure.edn/read-string {:default tagged-literal})))
#2019-10-18 08:45:35 rickmoynihan though it may well be good enough for basic use #2019-10-18 08:46:51 rickmoynihan but you’ll not get the metadata/protocol dispatch that REBL supports. REBL will only be able to walk the literal syntactic forms. #2019-10-18 08:46:52 vlaaad yes, datafy/nav will be much more limited since it happens in the same vm and not over wire, but rebl will still work. #2019-10-18 08:48:04 rickmoynihan Also it won’t be able to follow the repl etc #2019-10-18 08:58:37 vlaaad what do you mean follow the repl? #2019-10-18 08:59:18 rickmoynihan forms evaled into nrepl won’t appear in REBL #2019-10-18 08:59:39 rickmoynihan so only forms entered through REBL itself #2019-10-18 08:59:55 rickmoynihan unless i’m misunderstanding something #2019-10-18 09:03:48 vlaaad ah, yeah, I guess so, because rebl is a socket repl, and nrepl is not #2019-10-18 09:06:53 rickmoynihan I don’t think it’s anything to do with the difference between socket-repl and nrepl. I think it’s more to do with that they’re in different processes and can’t share the stdin stream to intercept. #2019-10-17 14:27:56 vlaaad (note, I used tagged-literal to be able to read what clojure can't usually read, for example clojurescript's js object notation: #js{:a 1}) #2019-10-18 10:00:35 timovanderkamp Hi, I have a dependency that suddenly broke. I am using https://clojars.org/amazonica and it results in the following error: Unable to resolve com.amazonaws/amazon-kinesis-client version: [1.7.5, 1.8.0). Any ideas what couldve caused this? #2019-10-18 10:13:25 timovanderkamp Thank you! #2019-10-18 12:54:10 alexmiller This may really need to be filed with sonatype for maven central #2019-10-18 11:32:14 rickmoynihan Is there a way to run clojure -Srepro -Sdeps {,,,} with a flag that prevents it merging the command line deps with a deps.edn file in the working directory? #2019-10-18 11:33:36 rickmoynihan i.e. an equivalent of Srepro but for the pwd’s deps.edn #2019-10-18 12:51:23 alexmiller No, but I am working on a release right now that may help you. What’s the use case? #2019-10-18 12:58:07 alexmiller Is it running a tool? #2019-10-18 13:23:45 rickmoynihan Hey… Yes it’s running a clj dev tool we have. #2019-10-18 13:25:48 rickmoynihan we pull in the tool over a git dep #2019-10-18 13:27:22 rickmoynihan how might the new release help? #2019-10-18 13:41:39 alexmiller the new feature lets an alias run with isolated rather than merged :deps with the project deps. that's similar but different than what you're wanting #2019-10-18 13:41:54 alexmiller do you want to completely remove the repo or just tweak it? #2019-10-18 13:42:55 alexmiller one thing I did happen to add in latest (not yet released) is the ability to remove a repo by setting it's name to nil #2019-10-18 13:43:30 alexmiller so if that was sufficient you could -Sdeps '{:mvn/repos {"the-server" nil}}' #2019-10-18 14:37:38 rickmoynihan completely removing would be fine #2019-10-18 14:38:06 rickmoynihan hmm interesting… that might work #2019-10-18 14:38:19 rickmoynihan I didn’t think of that #2019-10-18 14:39:03 alexmiller that will fail on current release, but should work on next release, coming. ... hopefully real soon #2019-10-18 14:39:44 rickmoynihan awesome — is real soon likely to be some time today? or next few days? #2019-10-18 14:40:12 rickmoynihan no need to promise 🙂 #2019-10-18 14:40:19 alexmiller hopefully today, although it sometimes takes a day for the brew pr to make it through #2019-10-18 14:40:37 rickmoynihan yeah stuff usually takes longer than you think #2019-10-18 14:41:28 rickmoynihan Anyway, thanks again for this… it’s very timely. #2019-10-18 14:42:28 rickmoynihan isolated deps would also be useful too — I’ve wanted that from time to time #2019-10-18 14:42:50 rickmoynihan but instead settled for refactoring stuff out of the core deps into aliases — the problem being the default case then suffers with extra alias baggage #2019-10-18 14:43:00 alexmiller yeah, it is particularly useful for tools #2019-10-18 15:37:38 zetafish Is there a way to prevent tools-deps to download maven-metadata.xml files? #2019-10-18 15:55:17 alexmiller currently, no. #2019-10-18 15:55:40 alexmiller are you looking for an "offline" mode? #2019-10-18 15:57:07 alexmiller if you already have needed versions locally, I'm actually not sure off the top of my head whether it will make any remote calls. #2019-10-18 15:58:05 alexmiller yeah, I don't think there's any way to do explicitly that right now #2019-10-18 15:59:38 zetafish I am using amazonica in deps.edn and explicitly exclusions the kinisis-client-library. It all worked until today that is. The annoying thing is that I do not even want to use the kinesis-client-library. #2019-10-18 16:00:44 alexmiller exclusions work on the library level, not version level. is it just running into trying to get an older version to do the check? #2019-10-18 16:01:46 zetafish Snippet of deps.edn:
com.amazonaws/aws-java-sdk-core {:mvn/version "1.11.521"}
com.amazonaws/aws-java-sdk-sts {:mvn/version "1.11.521"}
com.amazonaws/aws-java-sdk-s3 {:mvn/version "1.11.521"}
amazonica {:mvn/version "0.3.140"
:exclusions
[com.amazonaws/aws-java-sdk
com.amazonaws/amazon-kinesis-client]}
#2019-10-18 16:02:32 alexmiller yeah, I was more wondering how it manifests in a clj failure? #2019-10-18 16:03:04 zetafish % clj -Stree
Error building classpath. Unable to resolve com.amazonaws/amazon-kinesis-client version: [1.7.5, 1.8.0)
#2019-10-18 16:03:09 alexmiller I'm guessing amazonica relies on some particular older version and it's trying to find that, before it gets to the point of checking exclusions #2019-10-18 16:03:20 alexmiller yeah #2019-10-18 16:03:27 zetafish So something thinks that it should find [1.7.5, 1.8.0) and then later it is excluded anyway. #2019-10-18 16:03:34 alexmiller yeah #2019-10-18 16:04:12 alexmiller if you're just looking for a temporary workaround, you may be able to add it at the top level, with the version that's available, and exclude it there too #2019-10-18 16:04:59 alexmiller com.amazonaws/amazon-kinesis-client {:mvn/version "1.12.0" :exclusions [com.amazonaws/amazon-kinesis-client]} #2019-10-18 16:05:18 zetafish That looks really silly but let me try it 🙂 #2019-10-18 16:06:12 alexmiller nah, exclusions works only on children #2019-10-18 16:06:18 alexmiller so won't work #2019-10-18 16:07:21 alexmiller this will work... #2019-10-18 16:07:25 alexmiller clj -Sdeps '{:aliases {:x {:override-deps {com.amazonaws/amazon-kinesis-client {:mvn/version "1.12.0"}}}}}' -A:x #2019-10-18 16:07:42 alexmiller basically supply an alias that overrides the version to use to the one that happens to be available #2019-10-18 16:09:36 alexmiller actually, never mind, you'll still hit this #2019-10-18 16:09:39 zetafish Sadly no #2019-10-18 16:10:44 alexmiller have you checked any of hte maven mirrors? #2019-10-18 16:11:53 zetafish Not yet #2019-10-18 16:12:22 zetafish But I guess these mirrors will be updated to central at a certain moment, as they are mirrors. #2019-10-18 16:12:44 alexmiller maybe #2019-10-18 16:12:47 zetafish Or does each mirror keeps their owne maven-metadata #2019-10-18 16:13:11 alexmiller oh, not in metadata though #2019-10-18 16:16:14 alexmiller so you could try adding {:mvn/repos {"central" {:url ""}}} #2019-10-18 16:18:55 alexmiller that doesn't seem to work for me, but I'm not sure why #2019-10-18 16:20:27 zetafish Thanks. That will probably work (until it gets hit by 1.12.0 as well) #2019-10-18 16:27:15 alexmiller oh, that works if I rm the amazon-kinesis-client out of my ~/.m2 (it caches that source metadata) #2019-10-18 16:27:51 alexmiller maven central uploads have been giving me many weird errors over the last 24 hrs #2019-10-18 16:28:24 alexmiller might be something is awry there and an indexing job failed #2019-10-18 16:28:37 alexmiller did you file a ticket with sonatype? #2019-10-18 16:28:57 alexmiller I have an acct there already and I can do so if needed #2019-10-18 17:16:04 zetafish It would be great if you filed a ticket there. #2019-10-21 13:19:22 alexmiller looks like it's fixed now #2019-10-18 17:34:11 rickmoynihan our CI (travis) has .m2 cached… but it seems that running clojure for the first time on a single private git/dep in each build container refetches these deps each time:
Downloading: org/clojure/clojure/1.10.0/clojure-1.10.0.pom from
Checking out:
#2019-10-18 17:35:33 rickmoynihan I don’t know where they’re coming from… as hiccup is not a dependency of omni; and they already exist in .m2 #2019-10-18 17:36:25 rickmoynihan so it looks like tools.deps is refetching them #2019-10-18 17:42:09 alexmiller I can't really do anything without more info. need 1) deps.edn and 2) output of clj -Stree -Sdeps '{:aliases {:v {:verbose true}}}' -A:v #2019-10-18 17:42:09 alexmiller I can't really do anything without more info. need 1) deps.edn and 2) output of clj -Stree -Sdeps '{:aliases {:v {:verbose true}}}' -A:v #2019-10-18 19:03:29 alexmiller is this in response to request on @U06HHF230 - are you working together? or are there two things here? #2019-10-19 13:10:50 zetafish Wrong me #2019-10-21 10:08:29 rickmoynihan It’d be nice to support this, as it prohibits caching the deps in CI — which slows down the builds #2019-10-21 12:44:37 alexmiller Daily is the default so that matches what I would expect. Are any of these Snapshots? #2019-10-19 05:32:23 seancorfield seancorfield/depstar "0.3.4" -- now follows symlinks when copying directories (thanks to @mgrbyte for reporting the problem) #2019-10-20 21:05:32 andy.fingerhut This is probably too far off the beaten path to matter to anyone, but I noticed recently that Homebrew is also supported on Linux and Windows with WSL, and so tried out 'brew install clojure' on a Linux system. It seems to set things up in a way that is problematic for clojure finding the system-wide deps.edn file. I know there is the clojure install script for Linux, which is what I have always used before today, and will continue to do so. Just wanted to point this out in case anyone else tries it. #2019-10-20 22:37:58 alexmiller What’s the actual problem you saw? clojure actually no longer needs that file #2019-10-20 22:45:45 andy.fingerhut After brew install clojure completed with no apparent problems, I tried clojure -Sdescribe and got this message, which at the time stopped me from further experimentation: $ clojure -Sdescribe
cp: cannot stat 'PREFIX/example-deps.edn': No such file or directory
but I can try other things if you think it would be useful to know. #2019-10-20 22:47:07 andy.fingerhut A kind of unusual thing I noted about the results after brew install clojure completed was that Homebrew creates a /home/linuxbrew/.linuxbrew directory, where linuxbrew is a new directory, not the name of any user account on the machine, and puts all (or at least the ones I have seen) brew-installed files inside of that dir. #2019-10-20 22:49:29 andy.fingerhut Comparing to my works-better macOS install via brew install clojure earlier, it appears that the Linux clojure shell script has one occurrence of PREFIX that is not replaced with the install directory. #2019-10-20 23:54:21 andy.fingerhut I see what happened -- I tried this on an Ubuntu Linux system with minimal install, and it had no ruby executable installed. brew install clojure's install.sh script uses ruby to replace occurrences of the string PREFIX with the install dir. That step of install.sh script failed, leaving the clojure shell script in its original unmodified form. #2019-10-20 23:54:45 andy.fingerhut sudo apt-get install ruby, brew uninstall clojure, then brew install clojure made the install go better. #2019-10-20 23:55:10 andy.fingerhut Strange that the Homebrew install went well without installing ruby itself. #2019-10-20 23:57:36 seancorfield I thought brew used Ruby to do pretty much everything? #2019-10-20 23:58:05 seancorfield Perhaps the clojure recipe needs to be updated to explicitly state ruby is a dependency? #2019-10-20 23:58:36 seancorfield (I assume brew can be used to install ruby?) #2019-10-21 00:02:54 andy.fingerhut I think it can. Homebrew's first platform was macOS, I believe, where ruby has been installed by default since a long time ago (although I hear that might be changing in the latest macOS just released? Or perhaps the next one?) #2019-10-21 00:04:31 andy.fingerhut And probably if I had done a non-minimal install of Ubuntu 19.04 Linux, it might have included ruby as well. In any case, sudo apt-get install ruby definitely works if you do that before brew install clojure. It seems that the main Homebrew install instructions for Linux work without ruby being installed first, surprisingly enough. #2019-10-21 00:05:17 seancorfield (but it might be an "embedded" Ruby for its own use?) #2019-10-21 00:05:44 andy.fingerhut Ruby got installed after you did only this command? sh -c "$(curl -fsSL )" ? #2019-10-21 00:07:28 andy.fingerhut I will test on a fresh Linux VM and look at what happens a little more carefully while installing Homebrew. #2019-10-21 00:07:37 seancorfield Yeah, just confirmed it's an internal version of Ruby, just to run Homebrew! 😐 #2019-10-21 00:09:12 seancorfield Hmm, and to get Ruby you need patchelf which in turn requires Clang / gcc... OK, not going down that rabbit hole! 🙂 #2019-10-21 00:09:56 seancorfield So, could Clojure's brew recipe use something other than Ruby to do that textual change in the script? #2019-10-21 00:10:39 andy.fingerhut Even a minimal Ubuntu 19.10 Linux install includes gcc, and I remember brew installation installed patchelf early on before. #2019-10-21 00:11:28 seancorfield This is an earlier Ubuntu from an early WSL1 install (the joys of Insider builds) #2019-10-21 00:13:26 seancorfield (This is on my Mac running Win10 via Parallels with WSL as just a testing ground... maybe later I'll try it on my Win10 laptop although that's also WSL1 but I use it a lot more there... on this machine, I use Mac's Terminal for all my work) #2019-10-21 00:15:22 seancorfield Although on Windows now I'm using Scoop to install Clojure etc in PowerShell #2019-10-21 00:17:24 andy.fingerhut I believe I have followed the Homebrew Linux install instructions faithfully, and while it does install a ruby executable, it seems to be in a directory that does not end up on your command PATH after following the instructions, but some other directory. #2019-10-21 00:17:45 seancorfield Right:
> Yeah, just confirmed it's an internal version of Ruby, just to run Homebrew! #2019-10-21 00:20:09 andy.fingerhut It seems that perhaps there ought to be a way (or perhaps already is?) for a Homebrew install script to access that internal version. #2019-10-21 00:20:23 andy.fingerhut but that's a minor issue of polish, I think. #2019-10-21 00:22:51 seancorfield sed perhaps? #2019-10-21 00:37:15 andy.fingerhut Perhaps. It does appear that sed has a -i option on the couple of systems where I have checked, for easier overwrite-the-input-file-in-one-shell-command convenience. #2019-10-21 01:01:56 seancorfield Given that there's a cp just below it, why not use this directly sed "s;PREFIX;$prefix;" < clojure > $prefix/bin/clojure #2019-10-21 01:02:32 seancorfield So it would be mkdir ...
sed ...
cp ...
#2019-10-21 01:04:54 seancorfield I don't see much reason to modify the installed source files in place. #2019-10-21 01:05:09 seancorfield (I'll be interested to hear Alex's thoughts on this tomorrow) #2019-10-21 03:21:22 alexmiller sed has incompatible syntax between Mac and Linux #2019-10-21 08:04:15 dominicm It's the -i flag that's incompatible. You can easily avoid it. #2019-10-21 03:22:20 alexmiller If you can make it work, patch wanted #2019-10-21 04:59:18 andy.fingerhut Any doc page you would recommend on how to test modifications to a Homebrew install script? #2019-10-21 05:10:28 andy.fingerhut I have tested on a Linux machine using brew install --interactive clojure where I edited the install.sh script to replace ruby with ${HOMEBREW_RUBY_PATH}, and that worked. It used the Homebrew-internal installation of ruby. I will try that on a macOS system next to see if it works there, too. #2019-10-21 05:14:30 andy.fingerhut Yep, worked on a macOS system, too. That shell variable ${HOMEBREW_RUBY_PATH} was the full path name to a Homebrew-internal ruby executable on both a Linux system that had no other ruby installed, as well as on a Mac that had /usr/bin/ruby preinstalled before Homebrew was installed. #2019-10-21 05:21:15 andy.fingerhut I created a PR, even though they are likely not accepted for this project, just one convenient way to show the suggested change: https://github.com/clojure/brew-install/pull/5 . Feel free to close it. #2019-10-21 05:25:10 hiredman I think "brew-install" in the project name is a misnomer, and install.sh is expected to work on Linux sans homebrew #2019-10-21 05:26:45 andy.fingerhut The install.sh script that my PR suggests changing appears to be identical to a file with the same name that I found in both a Mac and Linux Homebrew installation when I installed the clojure Flask. #2019-10-21 05:28:25 andy.fingerhut The line my PR suggests changing is the one that was failing on a Linux system with no ruby executable. #2019-10-21 05:29:08 andy.fingerhut I mean, I am fairly new to this Homebrew stuff, so could be making any number of mistaken assumptions here, but giving it a shot. #2019-10-21 05:29:42 hiredman Sure, and it is also, unless I am mistaken, identical to install.sh which is used for installing on Linux systems which do not use homebrew #2019-10-21 05:30:18 andy.fingerhut There is another file named linux-install.sh in the same directory of that repo with install.sh, which uses install commands, that I suspect might be the non-Homebrew Linux install script. #2019-10-21 05:30:52 hiredman Ah, my mistake #2019-10-21 05:32:10 andy.fingerhut Always glad for another mind on the problem, though. Thanks for looking into it. #2019-10-21 05:39:31 hiredman I use Linux as my desktop os, and the idea of using homebrew to manage installed packages is so alien to me. I would consider just making the homebrew script exit with an error suggesting using the Linux install script on Linux. In which case you could depend on the bsd sed options and wouldn't need Ruby. But that is maybe a terribly unfriendly idea. #2019-10-21 05:41:31 andy.fingerhut The script works fine on macOS already today, since /usr/bin/ruby comes preinstalled (until either Catalina, I think, or maybe the next macOS release, when I have read it will no longer come preinstalled). With this change, it should continue to work on macOS, and start working on Linux. That doesn't mean anyone must use Homebrew on Linux -- just another option. #2019-10-21 08:05:35 dominicm I would just not sed in place, and use it to write elsewhere. #2019-10-21 08:05:44 dominicm -i is just a convenience flag with sed #2019-10-21 09:11:29 tianshu how to add a repository in deps.edn? #2019-10-21 09:25:28 dominicm There's a :mvn/repos key #2019-10-21 10:01:04 tianshu @dominicm @sogaiu thanks! #2019-10-21 10:23:16 kommen $ clojure
Error: Could not find or load main class clojure.main
$ clojure -Spath
Error: Could not find or load main class clojure.main
#2019-10-21 10:23:30 kommen $ brew info clojure
clojure: stable 1.10.1.478
The Clojure Programming Language
/usr/local/Cellar/clojure/1.10.1.478 (9 files, 21.1MB) *
Built from source on 2019-10-19 at 20:39:17
From:
==> Dependencies
Required: rlwrap ✔
==> Requirements
Required: java >= 1.8 ✔
==> Analytics
install: 3,096 (30 days), 11,637 (90 days), 49,842 (365 days)
install_on_request: 3,070 (30 days), 11,511 (90 days), 49,114 (365 days)
build_error: 0 (30 days)
#2019-10-21 10:33:21 kommen any body else having issues like this? #2019-10-21 10:34:06 dominicm Have you wiped your .m2? #2019-10-21 10:34:18 dominicm (I'm not suggesting you should) #2019-10-21 10:38:10 kommen no, I didn’t #2019-10-21 10:39:02 kommen I just tried to move .clojure/deps.edn somewhere else, suspecting a problem with it. but then I get: #2019-10-21 10:39:03 kommen $ clj
cp: PREFIX/example-deps.edn: No such file or directory
#2019-10-21 10:39:38 kommen with an ~/.clojure/deps.edn file only containing {} I guet the same Error: Could not find or load main class clojure.main error #2019-10-21 10:39:40 dominicm Ah. #2019-10-21 10:48:25 kommen since the -m flag of the java call was changed in this release, I suspect that this would be the problem: #2019-10-21 11:01:52 dominicm I don't think it is. I think your install failed. #2019-10-21 11:02:00 dominicm Given that you have issues with PREFIX #2019-10-21 11:03:09 kommen yes, you’re right #2019-10-21 11:03:29 kommen tried to reinstall already, but no luck #2019-10-21 11:04:06 dominicm Ruby powers the rewrite on Mac, that's all I know I'm afraid. Maybe there's a log somewhere? #2019-10-21 11:04:50 kommen woudn’t know where to look for a log #2019-10-21 11:05:03 dominicm It'll be a brew install log I'd guess. #2019-10-21 11:05:51 dominicm Yes #2019-10-21 11:13:04 kommen ok, so I manually fixed the install_dir= in /usr/local/bin/clojure and it works now, let’s see if this is only a problem with my machine or if some body else runs into it #2019-10-21 11:25:13 kommen for future reference: macOS 10.15 (19A583), $ ruby --version
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin15]
brew config
HOMEBREW_VERSION: 2.1.15
#2019-10-21 11:36:37 kommen ok, found the logs and this also explains it: #2019-10-21 11:36:38 kommen $ cat ~/Library/Logs/Homebrew/clojure/01.install.sh
2019-10-21 12:50:08 +0200
./install.sh
/usr/local/Cellar/clojure/1.10.1.478
truffleruby: the -p option is not implemented
#2019-10-21 11:48:32 dominicm Haha. Yes it does :) #2019-10-21 11:57:45 kommen @dominicm thanks for keeping company and pointing me in the right direction #2019-10-21 12:49:34 alexmiller Ok, so is there something to fix here or is this an issue mainly on your system? #2019-10-21 13:01:09 dominicm The problem is that truffle ruby is installed, and that didn't support -p for doing the substitution. Anyone who's playing with graal may run into this. #2019-10-21 13:01:51 alexmiller so the prior discussion here about using the brew install ruby impl would presumably also address that #2019-10-21 13:02:06 dominicm Yes. So would switching to sed. #2019-10-21 13:02:33 alexmiller -i has different incompatible syntax on mac and linux there #2019-10-21 13:02:45 alexmiller I used to use sed, then perl for a bit #2019-10-21 13:03:02 dominicm I know. But you can avoid using -i #2019-10-21 13:03:30 alexmiller I seem to remember there was some reason I didn't do that either, but it was a long time ago now #2019-10-21 13:08:30 dominicm Probably not a problem. Using brew means ruby anyway. The only impetus would be to unify the install scripts. #2019-10-21 13:11:21 plexus unfortunately some distros have stopped including ed... echo -e 'w clojure.bak\n1,$s^PREFIX^'$PREFIX'^\nwq' | ed clojure #2019-10-21 13:41:58 kommen I remember I once proposed a sed command which worked on linux and macos, but either my mind or jira search fails me #2019-10-21 13:43:01 andy.fingerhut This is exactly the same problem I found on Linux with Homebrew with no Ruby preinstalled. This will be an issue for all macOS Catalina systems, too, which also have no ruby preinstalled for the first time with macOS since it was introduced almost 20 years ago. #2019-10-21 13:43:04 kommen but Dominic is right, using brew means using ruby, so I guess it makes sense to be able to rely on ruby #2019-10-21 13:43:25 andy.fingerhut There is an issue open for Homebrew for this since June 2019 I found last night. #2019-10-21 13:44:10 andy.fingerhut I believe that the patch I submitted for the Clojure Homebrew installer will work for Catalina as well, because it will force the Clojure install.sh to use a ruby installed inside of Homebrew, rather than a system-wide one. #2019-10-21 13:44:31 andy.fingerhut But I do not have a macOS Catalina system to test on to be certain. #2019-10-21 13:46:19 alexmiller well, that certainly bumps it up my priority list #2019-10-21 14:55:50 markbastian Does anyone know if the immutant folks intended to let their domain expire? When I go to http://immutant.org/ there's just a domain expiration notification. #2019-10-21 15:05:00 dominicm This probably isn't the channel for that. @U06SGCEHJ you're associated I think? #2019-10-21 15:37:45 alexmiller definitely not the channel, but I'm not sure anyone is still working on immutant #2019-10-21 15:40:49 markbastian Ok, thanks for the info. I'll see if there's a better place. #2019-10-21 15:43:56 markbastian Oh, I just noticed I posted this in tools-deps, not clojure. Sorry! #2019-10-21 15:55:15 alexmiller tools.deps.alpha 0.8.578, clj 1.10.1.478 are now available - see #announcements for details #2019-10-21 15:58:06 alexmiller as a side note, there were changes made to the internal cli scripts between clj and tools.deps (clojure.tools.deps.alpha.script.make-classpath and clojure.tools.deps.alpha.script.generate-manifest). I doubt many people use any of that directly, but changes were made in clojure.tools.deps.alpha.script.make-classpath2 and clojure.tools.deps.alpha.script.generate-manifest2 to avoid any breakage. #2019-10-21 15:58:16 alexmiller no changes to tda apis #2019-10-21 22:13:05 ro6 Is anyone successfully using (add-lib ...) in a Cursive repl? I get a true back, but then "could not locate ... on classpath" when requiring the lib. #2019-10-21 22:17:28 cfleming @alexmiller The new :deps and :paths support in aliases is interesting. Am I correct in thinking that this would allow overriding the system deps.edn, e.g. to use a fork of Clojure? #2019-10-21 22:18:19 alexmiller It still has install deps on the chain #2019-10-21 22:18:34 alexmiller So, no difference there #2019-10-21 22:19:35 cfleming I see. Yes, re-reading the announcement I see I misread it - it’s more for tools to not require the project classpath instead. #2019-10-21 22:19:39 cfleming Thanks. #2019-10-22 06:33:29 dominicm Ooh, that would be very useful for pack #2019-10-22 07:11:47 andy.fingerhut Correction to something I said earlier -- macOS Catalina (the recently released version, earlier this month) does still include /usr/bin/ruby, perl, and python executables installed by default. I checked by installing it on one of my Macs in the house. Apple has announced that the next major release will not include them in the base macOS installation. #2019-10-22 07:15:38 andy.fingerhut I will soon be installing Homebrew on it, and brew install clojure, to see if it works for me. #2019-10-22 08:40:26 andy.fingerhut After installing a Java executable (I chose AdoptOpenJDK 11) and Xcode, and Homebrew, brew install clojure produced a working clojure command install on my macOS Catalina system. There is no need that I can see to change the install.sh script to make it work on that OS. #2019-10-22 08:41:24 andy.fingerhut It isn't clear to me why someone earlier had a problem there, unless perhaps /usr/bin wasn't in their path so that /usr/bin/ruby was not found, but that seems unlikely to be the reason. #2019-10-22 13:57:47 andy.fingerhut So at the time you tried upgrading the clojure package using brew upgrade clojure, you had a ruby executable in your command PATH that was earlier than /usr/bin/ruby? If so, the results you got makes sense to me. #2019-10-23 06:50:17 kommen @andy.fingerhut no, that is the part I didn’t understand but also didn’t investigate yet: how brew install clojure came to use truffleruby. truffleruby is not on my PATH… #2019-10-23 12:11:02 delaguardo Are you using graalvm with ruby component installed? #2019-10-23 12:13:48 kommen yes #2019-10-23 13:02:03 delaguardo probably you have graalvm bin directory in your path that overrides /usr/bin #2019-10-23 13:03:52 kommen no, that isn’t the case. also running ruby --version in the same shell as the brew commands shows that it uses a ruby version managed by rbenv #2019-10-23 13:39:06 delaguardo that is right, but during formula installation it loads new shell session with limited number of tools installed. The best way to check it - brew install --interactive clojure #2019-10-23 13:40:27 delaguardo As an example:
Install to this prefix: /usr/local/Cellar/clojure/1.10.1.478
/Users/delaguardo/.zshrc:29: command not found: starship
/Users/delaguardo/.zshrc:32: command not found: jenv
/Users/delaguardo/.zshrc:49: command not found: rbenv
clojure-tools % which ruby
/usr/bin/ruby
clojure-tools % ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]
#2019-10-23 13:42:02 andy.fingerhut In any case, I saw that Alex Miller has committed a change that I suggested where brew install clojure will run a shell script named install.sh such that it will run ruby using ${HOMEBREW_RUBY_PATH}, which Homebrew should consistently set to the full path name of its own internal ruby executable. This should make brew install clojure much more robust in the face of arbitrary user environments. #2019-10-23 13:42:45 andy.fingerhut I don't think that commit is deployed "to the field" yet, but on its way in that direction. #2019-10-23 13:44:52 kommen thanks @U04V4KLKC, that would explain it. I will check later #2019-10-23 13:46:00 kommen @andy.fingerhut thanks for the info, I’ll make sure to test brew install clojure with the next release #2019-10-23 13:46:24 delaguardo > In any case, I saw that Alex Miller has committed a change that I suggested where brew install clojure will run a shell script named install.sh such that it will run ruby using ${HOMEBREW_RUBY_PATH}, which Homebrew should consistently set to the full path name of its own internal ruby executable. This should make brew install clojure much more robust in the face of arbitrary user environments.
I’m not sure that this is the best solution. Now brew-install repo has two almost identical scripts that are doing the same thing but differently (using different tools) #2019-10-23 13:51:21 delaguardo Not in PR yet because I was hoping to find some time to build packages for other distribution platforms like debian’s ppa and rpm for centos #2019-10-24 05:47:43 amarjeet Hi, is there a way to provide credentials while using private git repo as a dependency in deps.edn? I want to use my private git repos in my project, but I am getting credentials related error. #2019-10-24 12:07:21 alexmiller What kind of credentials? #2019-10-24 12:07:56 alexmiller ssh is supported, user/pw is not currently #2019-10-24 14:51:14 amarjeet @U064X3EF3 For ssh, should I supply an additional key as :ssh in the deps map, in addition to :git/url and :sha? #2019-10-24 15:03:40 alexmiller no #2019-10-24 15:03:58 alexmiller we use the ssh-agent as a source for identity checking #2019-10-24 15:04:28 amarjeet oh okay #2019-10-24 15:05:11 alexmiller as a general principle, you will never put any kind of creds in deps.edn #2019-10-24 15:05:39 amarjeet Got it, makes sense. I will do it and try. Thanks much 🙂 #2019-10-25 10:13:51 borkdude can I specify a src folder for a local/root with a pom.xml (it's a boot project) #2019-10-25 10:14:10 borkdude somehow the source path doesn't show up in the pom.xml and when I run clojure -Spath it returns src/main/clojure #2019-10-25 10:14:36 borkdude the library is called dre.common and this is on my classpath: /Users/borkdude/dre/DocSearch/common/src/main/clojure #2019-10-25 10:40:24 borkdude Adding "../common/src" to the extra-paths works, but not sure if that's the right way #2019-10-25 12:30:47 delaguardo I have some problem trying to reposition library coordinates:
Library A’s deps.edn file
{:deps {B {:mvn/version "1"}}}
Library B’s deps.edn file
{:deps {C {:mvn/version "1"}}}
Library C’s deps.edn file
{:deps {}}
All version exists and can be resolved
Then I want to use git version of C library in A
Updated deps.edn for lib A:
{:deps {B {:mvn/version "1"}
C {:git/url ""
:sha "commit-sha"}}}
I’m expecting git version to take precedence in resolved dependency tree.
Instead I got an error:
Error building classpath. Unable to compare versions for C: {:mvn/version "1", :deps/manifest :mvn} and {:git/url "", :sha "commit-sha", :deps/manifest :deps, :deps/root "/home/user/.gitlibs/libs/C/commit-sha"}
Looks like a bug in clojure-cli for me. Happy to file a ticket if anybody can confirm (or agree) that this is a bug #2019-10-25 12:44:39 alexmiller there is a ticket already I think for being unable to compare versions for the same library across different coordinate types #2019-10-25 12:44:48 alexmiller still a todo #2019-10-25 12:45:17 alexmiller I have not read the above carefully, just going by the error #2019-10-25 12:49:17 delaguardo To summarize all above:
main problem that I’m trying to override dependency coordinate by adding it in top-level dep.edn. That is working fine within the same coordinate type (maven and git) but fails when I need git dependency to overtake maven. So yes, I think adding ability to compare different coordinate types should solve it #2019-10-25 12:54:47 alexmiller yes, this has not yet been implemented #2019-10-25 12:54:58 alexmiller although I've thought about it a lot and know what to do #2019-10-25 12:59:03 delaguardo nice! thanks #2019-10-25 12:59:35 delaguardo also I’m happy to help if you don’t mind to share you ideas around it #2019-10-25 13:55:03 gphilipp Not sure if it’s of any interest to anybody, but I’ve this small one-liner which pretty-print an EDN form in the clipboard:
clojure -Sdeps '{:deps {mvxcvi/puget {:mvn/version "1.2.0"}}}' -e "(require '[puget.printer :as puget]) (puget/cprint (read-string (first *command-line-args*)))" $(mktemp) "$(pbpaste)"
I’m not sure why the first arg needs to be a file since I’m using the -e option (maybe https://clojure.atlassian.net/browse/TDEPS-56) but well, it works. #2019-10-25 14:46:45 borkdude @U050S183X if you're interested in a tool with milliseconds startup time:
$ jet --pretty <<< "$(pbpaste)"
{:a 1, :b 2, :c 3}
#2019-10-25 14:56:05 borkdude jet doesn't have colors yet. #2019-10-25 18:38:43 andy.fingerhut Is there some kind of option to clj/clojure commands to cause them to show the java commands they execute? #2019-10-25 19:06:47 seancorfield bash -x clojure <args> #2019-10-25 19:07:14 seancorfield (that shows "everything" but it does show the java command as well 🙂 ) #2019-10-25 19:07:49 seancorfield bash -x clojure
...
+ exec /usr/bin/java -Dclojure.libfile=.cpcache/1562564545.libs -classpath src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar clojure.main
Clojure 1.10.1
user=>
#2019-10-25 19:08:28 seancorfield (you can't do bash -x clj ... because it doesn't pass -x into clojure when it runs it via rlwrap) #2019-10-25 19:08:58 andy.fingerhut Huh, didn't know about -x option to bash. Looks generally useful for all bash scripts. #2019-10-25 19:09:10 seancorfield Yeah, I use it a lot for debugging scripts. #2019-10-25 19:54:42 alexmiller ditto :) #2019-10-25 20:22:28 andy.fingerhut I mean, I knew about the set -x command by editing my own bash scripts to debug them, and sometimes did that on things owned by root if I had the permission, but nice that it can also be done without editing the script. #2019-10-25 23:02:58 sogaiu is there something comparable for powershell? #2019-10-29 05:07:12 johan Hello, did anyone successfully use nRrpl and repl-y together to be launched via a clojure -A:repl ? #2019-10-29 05:33:54 seancorfield @johan178 Something like this? clj -Sdeps '{:deps {reply {:mvn/version "RELEASE"}}}' -e "((requiring-resolve,'nrepl.server/start-server))" -m reply.main
That will start an nREPL server on a random port (and report it) as well as starting REPL-y. #2019-10-29 05:35:05 seancorfield Then you can connect into that server from other clients. I just tried connecting in with Leiningen to check it worked. #2019-10-29 05:35:33 seancorfield You could provide a specific port number if you wanted in the call to start-server. #2019-10-29 06:36:54 johan :nrepl {:extra-deps {nrepl {:mvn/version "RELEASE"}
reply {:mvn/version "RELEASE"}
cider/cider-nrepl {:mvn/version "RELEASE"}}
:main-opts ["-e" "((requiring-resolve,'nrepl.server/start-server))"
"-m" "reply.main"] }
I got an alias working.
Trying to get nrepl midlerware (cider-nrepl) working #2019-10-29 11:19:56 borkdude (fwiw I don't start a REPL in terminal, I just connect to it from my editor) #2019-10-31 05:23:09 maxp Is usage of "RELEASE" / "LATEST" in :mvn/version documented somewhere? #2019-10-31 05:36:28 seancorfield @maxp They are not really recommended. #2019-10-31 05:38:19 seancorfield They are not supported in Maven 3, I believe. They are only available in Maven 2 based tools. #2019-10-31 05:39:23 seancorfield RELEASE means the most recent non-snapshot release, so it can be an alpha, for example, or a "gold" version. LATEST means the most recent -SNAPSHOT release which can change frequently. #2019-10-31 05:41:21 seancorfield That said, I use RELEASE for dev-only tooling, as a "convenience" for it always being up-to-date but with the caveat that it could of course break any day. I never use LATEST. #2019-10-31 05:41:38 seancorfield Does that help @maxp? #2019-10-31 06:36:04 maxp @seancorfield - yes. Thank you/ #2019-10-31 16:31:53 drewverlee Its possible to specify multiple deps files and have later ones override earlier ones right? #2019-10-31 16:35:25 alexmiller there are multiple deps files that override, but that is not an open set #2019-10-31 16:35:40 alexmiller the deps chain is: install deps.edn (now a resource in tools.deps.alpha), user deps.edn (~/.clojure/deps.edn), project deps.edn, -Sdeps #2019-10-31 16:36:29 alexmiller if you're using tools.deps directly, you can of course do whatever you like #2019-10-31 16:36:42 alexmiller it has tools for loading and merging deps.edn files #2019-10-31 16:42:14 drewverlee Thanks alex. #2019-10-31 16:58:41 drewverlee so if you want to override a project dep it would be something like "-Sdeps {:aliases {:foo {:override-deps ...}}} -A:foo" #2019-10-31 17:07:11 alexmiller you don't need an alias for that #2019-10-31 17:07:38 alexmiller can just -Sdeps '{:deps {the/lib {:mvn/version "whatever"}}}' #2019-10-31 17:08:02 alexmiller :deps merge in #2019-10-31 17:08:10 alexmiller you can do what you said too though #2019-10-31 17:12:10 drewverlee makes sense. #2019-11-03 14:19:01 athomasoriginal I love cofield’s dot-clojure repo: https://github.com/seancorfield/dot-clojure/blob/master/deps.edn. I would like to make an alias in my own global deps.edn file that will use the latest version of ClojureScript e.g.
{:aliases
{:cljs
:extra-deps {org.clojure/clojurescript {:git/url ""
:sha "6431e6435ad9fc85e766752f4d6939ff978db5a5"}}
:main-opts ["--main" "cljs.main" "--repl-env" "node" "--repl"]}}}
But when I call the above, I get Error building classpath. Destination path "clojurescript" already exists and is not an empty directory. What am I missing here? #2019-11-03 16:29:48 athomasoriginal I did a little digging and was able to resolve my issue by executing the following steps:
1. Delete ~/.gitlibs/_repos/github.com/clojure/clojurescript (which was empty)
2. Run clj -A:cljs again
Now everything seems to be working again. Is there a command in the tools.deps API that can do this automatically? #2019-11-03 16:30:34 alexmiller I’ve seen a few people hit this but I have not been able to reproduce it and don’t have a ready fix for it #2019-11-03 16:42:51 athomasoriginal Fair. I tried to setup a minimal repro based on what my previous dir structure looked like, but can’t get it right. The best I found is this other interesting scenario
1. Successfully run clj -A:cljs as I have in the code snippet above (then stop the repl)
2. delete content of ~/.gitlibs/libs/org.clojure/clojurescript + ~/.gitlibs/_repos/github.com/clojure/clojurescript (both clojurescript still exist, but they are empty)
Now when I run clj -A:cljs I get the following error
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate cljs/main__init.class, cljs/main.clj or cljs/main.cljc on classpath.
Full report at:
/var/folders/qc/mv_jl6857850zkr0qv6gj89m0000gn/T/clojure-5520844173070466309.edn
The above is resolved by deleting .cpcache #2019-11-03 16:44:51 athomasoriginal The above seems odd, but that seem to be how my local looked at the time. #2019-11-03 18:36:58 borkdude fwiw I had a similar problem recently (went away after deleting .gitlibs and/or .cpcache) #2019-11-03 20:55:20 seancorfield @tkjone Or perhaps putting -Sforce on that clj command, to force it to rebuild the cache? #2019-11-03 20:59:17 sogaiu i've been doing -Sforce more often than deleting .cpcache with some success recently -- iiuc this will leave older bits in .cpcache if you want to investigate issues later, helpful when reporting issues perhaps? #2019-11-03 21:03:58 seancorfield Yeah, you get a separate set of cached files for each alias combination you use, so it's nice to keep the ones that work fine and just override the ones that don't, via -Sforce. #2019-11-04 11:05:45 vlaaad Another use-case for https://clojure.atlassian.net/browse/TDEPS-116 — I want to create a library that has different code for different jdk versions. I do it using 2 aliases:
{:jdk8 {:extra-paths ["src/jdk8"]}
:jdk11 {:extra-paths ["src/jdk11"]}}
It would be much easier to depend on such library when you can specify extra alias when declaring a dependency to it #2019-11-04 11:45:24 vlaaad I think I found a bug: :paths in aliases are not picked up when there are also :deps present. Repro:
deps.edn:
{:paths ["src" "jdk11"]
:aliases {:jdk8 {:paths ["src" "jdk8"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}}}}}
clj -Srepro -Sforce -Spath output:
src:jdk11:/home/vlaaad/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/home/vlaaad/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/vlaaad/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
clj -A:jdk8 -Srepro -Sforce -Spath output:
src:/home/vlaaad/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/home/vlaaad/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/vlaaad/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
notice: second output does not have jdk8 folder on a classpath #2019-11-04 11:50:08 vlaaad #2019-11-04 11:50:08 vlaaad #2019-11-04 13:16:05 stijn @vlaaad you have a typo in the deps.edn, :deps is not inside the :jdk8 alias. #2019-11-04 13:17:20 vlaaad what, where? it is! #2019-11-04 13:18:02 vlaaad oh, you are right #2019-11-04 12:11:55 dominicm In maven that's a classifier I think. Although jdk11 had a system for having jdk-specific files added. #2019-11-04 12:22:06 dominicm @vlaaad jep238: multi release jars #2019-11-04 12:23:51 dominicm It would be cool if deps.edn supported MRJAR for fetching dependencies. #2019-11-04 13:01:33 alexmiller I don’t think mrjars have any effect on deps? That’s really a JVM thing? #2019-11-04 13:01:59 alexmiller On the problem above, what version of clj are you on? #2019-11-04 13:06:58 vlaaad latest, 1.10.1.148 #2019-11-04 13:10:11 vlaaad @dominicm will jep 238 work with clj files as well? is it okay to make library muti-release?.. #2019-11-04 13:10:54 dominicm I see no reason why not. Try it and report back! #2019-11-04 13:12:39 vlaaad I think jep238 might not be a fitting solution for me, because I also need different dependencies for different java versions... #2019-11-04 13:29:59 alexmiller Yeah, won’t do that, but should work for clj files as they are loaded as resources, just like class files #2019-11-04 13:32:23 vlaaad yeah, makes sense #2019-11-04 13:34:14 alexmiller This is really where maven uses classifiers and jdk properties #2019-11-04 13:10:33 dominicm I was thinking of fetching the appropriate pom.xml from the jar based on jdk version #2019-11-04 13:32:08 alexmiller I don’t think that’s a thing #2019-11-04 13:32:52 alexmiller Although I have not looked at all at the maven support for mrjars #2019-11-04 13:35:48 dominicm It doesn't look like it is. It could be for local/root on a jar, but for maven a separate fetch is made for the pom, completely unaware of the jdk #2019-11-04 13:39:48 vlaaad is there a way to exclude dependency using alias? #2019-11-04 13:41:22 alexmiller Not right now #2019-11-04 13:54:05 vlaaad sad, I'm running out of ideas how to add jdk8 support without breaking git dependencies to cljfx 😞 #2019-11-04 13:56:20 vlaaad maven is easy, I can build poms with different deps and source paths and publish them either as different artifacts, or with different classifiers/activations #2019-11-04 13:57:32 vlaaad but I already have jdk11-specific deps in root deps.edn, and it seems there is no way to exclude them when using git dependency? #2019-11-04 13:59:26 alexmiller you could try using a lib with nil coordinate - that will merge over, but I don't remember how tools.deps handles it (could easily not work, I just don't remember) #2019-11-04 14:01:26 vlaaad using :override-deps? #2019-11-04 14:01:50 alexmiller yeah #2019-11-04 14:04:21 vlaaad doesn't seem to work:
given this deps.edn:
{:paths ["src" "jdk11"]
:deps {org.openjfx/javafx-base {:mvn/version "13"}}
:aliases {:jdk8 {:paths ["src" "jdk8"]
:override-deps {org.openjfx/javafx-base nil}}}}
running clj -A:jdk8 -Srepro -Sforce -Spath still includes javafx-base:
src:jdk8:/home/vlaaad/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/home/vlaaad/.m2/repository/org/openjfx/javafx-base/13/javafx-base-13.jar:/home/vlaaad/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/vlaaad/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/home/vlaaad/.m2/repository/org/openjfx/javafx-base/13/javafx-base-13-linux.jar
#2019-11-04 14:05:12 vlaaad and using :deps in alias seems to interfere with :paths so jdk8 path is not seen, as reported earlier #2019-11-04 14:05:33 alexmiller yeah, I'm going to look at that in a bit once I'm past my morning calls #2019-11-04 14:06:42 alexmiller that's surprising to me #2019-11-04 14:07:04 alexmiller could you also post me your clj -Sdescribe and whether you have anything in ~/.clojure/deps.edn? #2019-11-04 14:09:35 alexmiller wait, you said latest but 1.10.1.148 is not a thing, should be 1.10.1.478 #2019-11-04 14:10:03 vlaaad I'm using -Srepro, so ~/.clojure/deps.edn shouldn't be an issue? #2019-11-04 14:10:32 vlaaad I'm using 1.10.1.478, it was a typo #2019-11-04 14:10:38 alexmiller ah, gotcha #2019-11-04 14:10:45 alexmiller on both #2019-11-04 14:11:38 vlaaad clj -A:jdk8 -Srepro -Sforce -Sdescribe
{:version "1.10.1.478"
:config-files ["/usr/local/lib/clojure/deps.edn" "deps.edn" ]
:config-user ""
:config-project "deps.edn"
:install-dir "/usr/local/lib/clojure"
:config-dir "/home/vlaaad/.clojure"
:cache-dir ".cpcache"
:force true
:repro true
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ":jdk8"}
#2019-11-04 14:16:25 alexmiller oh, this is the expected behavior #2019-11-04 14:16:44 alexmiller the :paths and :deps in an alias are new with the newest release and REPLACE the project :paths and :deps #2019-11-04 14:17:07 alexmiller if you want to add, you need :extra-paths or :extra-deps #2019-11-04 14:17:28 vlaaad I want to replace both #2019-11-04 14:17:50 alexmiller I thought your complaint above was that the project :paths was not included? #2019-11-04 14:18:22 alexmiller re-reading what you said above... #2019-11-04 14:18:22 vlaaad No, replacement paths are not picked up #2019-11-04 14:51:31 alexmiller ah, it is a bug and I see the fix #2019-11-04 15:14:41 alexmiller and there is a test that should have caught this, but the test is missing the is, so it didn't fail :) #2019-11-04 15:14:55 alexmiller programming is hard, y'all #2019-11-04 15:31:36 andy.fingerhut eastwood will probably warn about such tests, if it works on your code base at all. #2019-11-04 16:03:12 kszabo clj-kondo as well #2019-11-04 16:09:57 vlaaad thanks a lot! #2019-11-04 16:11:31 alexmiller thanks for a concise and useful report! :) #2019-11-04 16:07:03 alexmiller I didn't that see that with clj-kondo, but I'm not running latest version #2019-11-04 16:08:22 borkdude depends on the example. what's the code? #2019-11-04 16:10:09 borkdude found it. yes, that's a case I want to improve. currently it only checks the direct children of (deftest ...) like (deftest foo (= 1 2)), but these cases were inside a let: https://github.com/borkdude/clj-kondo/issues/462 #2019-11-04 16:30:41 andy.fingerhut There are also typically many is/are expressions inside of testing subexpressions inside of deftest, or let/testing/let/... etc. #2019-11-04 16:31:22 borkdude yeah. does eastwood test for that btw? #2019-11-04 16:34:47 andy.fingerhut Eastwood uses tools.analyzer to analyze everything, then looks for all expansions of is forms, I believe. #2019-11-04 16:35:08 andy.fingerhut It is one way to do it, but can be painful to recognize the expansions of is, which is an odd macro. #2019-11-04 16:35:35 borkdude but the point is to detect that is was forgotten. so it warns when you have a deftest without a single occurrence of is? #2019-11-04 16:35:54 borkdude globally? or does it also point to the location where the is was expected? #2019-11-04 16:36:16 andy.fingerhut I'd have to go back to the code to check, but I think it warns about all expressions that are not likely to be called only for their side effects, that do not have is or are around them. #2019-11-04 16:37:27 andy.fingerhut I'll follow up in #eastwood channel after checking the code for a little bit. #2019-11-04 17:12:14 borkdude I guess even clojure.test could have built-in warnings like: you're defining a deftest but you never called is.
Or: you called testing, but you never called is. #2019-11-04 17:50:26 alexmiller But it’s difficult to verify that beyond lexical scope #2019-11-04 17:54:32 borkdude I'm not sure how clojure.test works internally, but is there not some kind of registry to which test an assertion belongs? #2019-11-04 17:56:16 borkdude There are all kinds of hooks. Maybe it's possible to verify during one of those hooks that nothing got actually checked when running a test. #2019-11-04 17:57:09 alexmiller Possibly #2019-11-04 17:57:57 alexmiller I’ve also written tests with negative assertions in code that shouldn’t run in the past. In those cases, even no assertions is correct #2019-11-04 17:58:15 alexmiller But those are prob rare #2019-11-04 18:02:07 borkdude that's meta: assert on a hook that the number of assertions is zero #2019-11-04 18:07:31 alexmiller you could also write those kind of tests as a zero? check on a "reached" count in an atom I suppose so there would always be an assertion #2019-11-04 18:08:40 borkdude right #2019-11-04 18:10:01 alexmiller @U08E8UGF7 the build here is a Maven build (as with all contrib builds) so not trivial to integrate #2019-11-04 18:10:50 alexmiller ironically of course, given it's tools.deps #2019-11-04 18:18:19 borkdude I would like to see this in clojure.test, but I think the chances are fairly slim it would ever get in #2019-11-04 18:18:53 borkdude but as a library it could work #2019-11-04 18:21:07 kszabo this is one of the many reasons why we use kaocha, of course a static analysis check would be better as well. The negative assert feature could be fixed with dedicating another name for it like (clojure.test/never-reached!) which macros could check for. If I got the negative assert idea correct #2019-11-04 19:23:58 kszabo cool! so productive, @U04V15CAJ 🙂 Can you pretty please add a deps.edn as well for us tools.deps users. #2019-11-04 19:24:26 borkdude on it 🙂 #2019-11-04 19:28:59 borkdude @U08E8UGF7 it's in there now. I'm trying to make it work under CLJS as well, but registering the multimethod doesn't seem to work #2019-11-04 20:50:21 borkdude Now also works in ClojureScript. #2019-11-05 04:05:01 seancorfield Taking a look... #2019-11-05 04:09:47 seancorfield There's definitely something odd going on: PS C:\var\www\async.http\sketch> clojure -R:a -R:b -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.xml\0.2.0-alpha6\data.xml-0.2.0-alpha6.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;C:\Users\seanc\.m2\repository\org\clojure\data.codec\0.1.0\data.codec-0.1.0.jar
PS C:\var\www\async.http\sketch> clojure -A:a -A:b -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.xml\0.2.0-alpha6\data.xml-0.2.0-alpha6.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;C:\Users\seanc\.m2\repository\org\clojure\data.codec\0.1.0\data.codec-0.1.0.jar
PS C:\var\www\async.http\sketch> clojure -Sforce -A:a -A:b -Spath
WARNING: Specified aliases are undeclared: [:a ]
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar
PS C:\var\www\async.http\sketch> clojure -Sforce -R:a -R:b -Spath
WARNING: Specified aliases are undeclared: [:a ]
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar
PS C:\var\www\async.http\sketch>
#2019-11-05 04:10:50 seancorfield I got the same results from -R and -A tho' in all cases (including -A:a:b and -R:a:b, not shown) but with -Sforce, the first alias is somehow not found... #2019-11-05 04:12:08 seancorfield After running that -Sforce variant, running the command without -Sforce is now behaving as if :a was not included. #2019-11-05 04:12:38 alexmiller that extra space in [:a ] is probably important - I presume that's causing the alias not to be found there #2019-11-05 04:13:18 seancorfield PS C:\var\www\async.http\sketch> clojure -R:a:b -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar
PS C:\var\www\async.http\sketch> clojure -R:b:a -Spath
src;C:\Users\seanc\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\seanc\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\seanc\.m2\repository\org\clojure\data.xml\0.2.0-alpha6\data.xml-0.2.0-alpha6.jar;C:\Users\seanc\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\seanc\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;C:\Users\seanc\.m2\repository\org\clojure\data.codec\0.1.0\data.codec-0.1.0.jar
Note that swapping the aliases produces the correct result (since it's treated as a new invocation) #2019-11-05 04:15:27 seancorfield Then I ran it with -Sforce -R:a:b and got the correct behavior back. So I'm suspecting a problem with parsing options if -Sforce is used? #2019-11-05 04:18:09 seancorfield Yeah, almost certainly something about -Sforce messing up alias parsing or combining somehow based on further experiments. #2019-11-05 04:20:29 seancorfield That's with PS 1.10.1.469 BTW. #2019-11-05 04:22:59 seancorfield And it's peculiar to the PS script. The exact same sequences of commands on Linux just work. #2019-11-05 04:25:03 seancorfield (how did he get .483 on Windows? I tried scoop update clojure and it thinks .469 is the latest... maybe the wiki page has not been updated?) #2019-11-05 04:25:15 alexmiller I updated the wiki page #2019-11-05 04:26:08 alexmiller I don't anything about the scoop thing though #2019-11-05 04:26:20 seancorfield Yeah, the scoop script is supposed to use that link in that page to pick up a newer version... #2019-11-05 04:26:56 alexmiller I wouldn't expect these to differ in this though #2019-11-05 04:27:06 alexmiller so it's likely same in 483 #2019-11-05 04:27:19 seancorfield Fair enough. Now I'm still puzzled about the scoop update process 🙂 I'll open an issue... #2019-11-05 04:27:27 alexmiller I assume these are bugs in the ps script #2019-11-05 04:42:48 seancorfield Interesting. Now I've managed to get this in a clean folder without using -Sforce: PS C:\var\www\async.http\sketch> clojure -A:a -A:b
WARNING: Specified aliases are undeclared: [:a ]
Clojure 1.10.1
user=>
#2019-11-05 04:43:04 seancorfield That makes a bit more sense given the report on ask... #2019-11-05 04:46:55 alexmiller [:a ] has an extra space which I assume is the problem #2019-11-05 04:48:13 seancorfield I suspect the default conversion for multi-element collections is to have spaces between the elements. #2019-11-05 04:48:49 alexmiller the tools.deps code splits on colons, so could easily end up with keywords with spaces in them #2019-11-05 04:48:52 seancorfield So $ToolsArgs += "-R$($ResolveAliases -join '')"
etc #2019-11-05 04:51:27 alexmiller both $ there? #2019-11-05 04:51:39 alexmiller guess so #2019-11-05 04:53:21 seancorfield Yup, that's the fix. I just updated my .psm1 script to have that and it works as expected now. #2019-11-05 04:53:53 seancorfield All of those $ToolsArgs += lines need that... #2019-11-05 04:55:04 seancorfield Looks good! #2019-11-05 04:56:33 alexmiller ok, there's a windows build out there at .486 now #2019-11-05 04:57:54 seancorfield Wiki page updated? #2019-11-05 05:00:44 alexmiller yes #2019-11-05 05:02:55 alexmiller heading to bed, thx for the help #2019-11-05 05:37:16 andy.fingerhut I tried installing Homebrew then brew install clojure on a couple of Ubuntu Linux VMs with different versions, and they both installed version 1.10.1.478 still. brew update clojure on my Mac upgraded to 1.10.1.483, so perhaps Homebrew Linux is taking longer to update package info somewhere? #2019-11-07 03:23:21 seancorfield @U0CMVHBL2 FWIW, I installed brew on Ubuntu (actually WSL on Windows 10), and then brew install clojure installed 1.10.1.483. #2019-11-07 03:32:25 andy.fingerhut Cool. I will check on an Ubuntu Linux system again soon and report back, to see if it gets the latest version now. #2019-11-05 05:39:22 andy.fingerhut Strangely enough, though, brew install clojure worked fine on those Linux machines, with no problem finding a working version of ruby executable to do what it needed to install the clojure thingy (replace "thingy" with correct Homebrew term, whatever that is) #2019-11-05 12:45:00 alexmiller Formula #2019-11-05 12:45:54 alexmiller Maybe Linux brew is downstream somehow #2019-11-05 21:53:56 seancorfield Another Windows glitch? Can someone else on Windows PowerShell confirm/deny this for me? PS C:\Users\sean> clojure -Sdescribe
{:version "1.10.1.486"
:config-files ["C:\Users\sean\scoop\modules\ClojureTools\deps.edn" "C:\Users\sean\.clojure\deps.edn" "deps.edn"]
:config-user "C:\Users\sean\.clojure\deps.edn"
:config-project "deps.edn" when run in a folder that does not have it still reports it in describe and as the :config-project path. The macOS/Linux version displays it as :config-project (which still seems wrong) but correctly omits it from the :config-files path. #2019-11-05 21:57:36 alexmiller The behavior in Mac is right - it’s telling you where it’s looking #2019-11-05 21:57:52 alexmiller config-files is not actually used now #2019-11-05 21:58:59 alexmiller The Clojure code will do the right thing if it’s missing #2019-11-05 22:02:39 seancorfield OK, so a small glitch in the PS script but not an important one. #2019-11-05 22:03:20 seancorfield macOS (! 720)-> clojure -Sdescribe
{:version "1.10.1.483"
:config-files ["/usr/local/Cellar/clojure/1.10.1.483/deps.edn" "/Users/sean/.clojure/deps.edn" ]
:config-user "/Users/sean/.clojure/deps.edn"
:config-project "deps.edn"
#2019-11-05 22:03:56 alexmiller Seems ok to me #2019-11-05 22:07:14 seancorfield Given that tooling should no longer be running clojure -Sdescribe via (clojure-env), and such tooling was broken on Windows anyway(!), I agree that nothing really needs to be done. It's just slightly odd to look at. #2019-11-05 22:09:36 alexmiller config-files is not even passed to the script anymore #2019-11-06 14:14:55 Michaël Salihi Hello everybody, I would to know if I can kick off a one command line simple http server with Clojure like Python does python3 -m http.server ?
With tools.deps for eg. ? clj -Sdeps '{:deps {some/clojars {:mvn/version "x.x.x"}}}' -m http.server #2019-11-06 17:30:01 Michaël Salihi Perfect ! Thx @U2J4FRT2T #2019-11-06 15:27:40 dominicm I think there was a snippet for that kicking around #2019-11-06 17:26:24 rickmoynihan @dominicm: I’m currently using the capsule builder for pack.alpha, but running into annoyances it brings wrt the JVM trampoline, tunnelling args and also wanting to extend the classpath of the “uber.jar” from the outside via java -cp uber.jar:./my-file ,,,. Is the one-jar builder likely to cause me less friction? #2019-11-06 17:27:29 rickmoynihan their docs describe the problem, but not really their solution other than as a custom classloader that redirects requests. #2019-11-06 18:04:05 dominicm Haven't used either in this way. #2019-11-06 18:04:44 dominicm OneJAR will likely work well for this case, based on my recollection of its function. It'll delegate. #2019-11-07 03:24:38 seancorfield How to confuse yourself: in Ubuntu bash on Windows (via WSL 1), type scoop update clojure, and wonder why, after completing successfully, when you type clojure -Sdescribe it shows the version you previously had installed... #2019-11-07 03:25:36 seancorfield ...then switch to PowerShell and run clojure -Sdescribe and realize that it was updated: Windows figured out scoop was a Windows command, even tho' you typed it into Ubuntu, and ran it on Windows anyway... 👀 #2019-11-07 03:33:24 andy.fingerhut I'm sticking with Linux in a VM, if I ever use a Windows machine regularly 🙂 #2019-11-07 03:34:11 seancorfield WSL 2 is a full VM-based approach. I'm stuck with WSL 1 for... reasons... on both of my Windows systems. #2019-11-07 03:34:33 seancorfield (I run Windows 10 on my Mac, BTW, and pretty much only use macOS for Terminal and Docker!) #2019-11-07 04:09:12 drewverlee so the command here: clj -A:pack mach.pack.alpha.jib --image-name --image-type registry -m my.main
from https://github.com/juxt/pack.alpha. seems to suggest that by passing -m its going to record that as the main namespace. but whats happening is that its "running" that namespace. which makes sense given that passing -m to clj does just that.
what i'm i missing? #2019-11-07 04:10:39 drewverlee hmmm #2019-11-07 04:12:27 drewverlee actually that might not be the issue. is mach.pack.alpha.jib a script/function im passing clj and the rest are the args? #2019-11-07 04:12:38 drewverlee yea, thats probably right #2019-11-07 04:13:27 drewverlee my issue was that i told the pack alias to run the main function. #2019-11-07 07:41:06 andy.fingerhut I tried on an Ubuntu 16.04 Linux machine (freshly installed, latest updates from Canonical as of today), to install Homebrew for Linux, then brew install clojure. It installed the latest version 1.10.1.483 which brought up a REPL just fine. The Linux VM I tested on had no previous installation of ruby, clojure, nor Java. brew install clojure installed them all automatically as dependencies, choosing AdoptOpenJDK 8 for the JDK (plus some things that surprised me, but didn't hurt -- e.g. gcc seemed weird) #2019-11-07 10:56:18 robert-stuttaford Probably a very-FAQ, but googling didn't help me at all. how do teams deal with sharing deps.edn files that use :local/root? one option is to symlink the common code folder on everyone's machines, but i'm wondering if the lib itself supports something out of the box #2019-11-07 10:58:28 dominicm It depends 🙂 #2019-11-07 10:58:52 dominicm If they're released separately, then I wouldn't use :local/root #2019-11-07 10:59:09 robert-stuttaford this is just to support a short period of active new development #2019-11-07 10:59:43 robert-stuttaford release will use the usual s3 maven repo nonsense, until we can get everything transitioned to deps, and then we'll use the git sha thing #2019-11-07 11:04:25 robert-stuttaford is there inbuilt capability to resolve local relative paths, or is symlinks the way? #2019-11-07 11:06:27 dominicm just as a workflow thing for what I do: as someone doing multi-repo development I personally setup aliases in my ~/.clojure/deps.edn file. #2019-11-07 11:06:49 dominicm Some of my colleagues don't do any cross-development, so they don't have to care. When I do the cross development, I just activate the extra profile(s) on launch. #2019-11-07 11:06:58 dominicm relative paths do work though. #2019-11-07 11:08:58 robert-stuttaford ah, i tried relative to home ~/Code/... but not relative to folder ../..., going to try that now #2019-11-07 11:09:04 robert-stuttaford thanks Dominic! #2019-11-07 11:09:38 dominicm Yeah, no support for ~. I think it's been discussed & rejected. .. works though, used heavily in Edge. #2019-11-07 11:16:43 robert-stuttaford nice! #2019-11-07 22:00:44 drewverlee if your projects deps.edn uses git/url to point at another project, does it have to also have a deps.edn? or would a project.clj work? i dont think so, but its been a while. #2019-11-07 22:01:11 kszabo the former #2019-11-07 22:01:49 drewverlee sorry, could you clarify? i asked my question awkwardly. It must have a deps? #2019-11-07 22:03:31 alexmiller it must have either a deps.edn or a pom.xml #2019-11-07 22:03:46 alexmiller or neither #2019-11-07 22:04:14 alexmiller deps does not currently support project.clj as a manifest as there is no good way to canonically load it and find the deps without basically running all of lein #2019-11-07 22:04:36 drewverlee right, thanks alex. #2019-11-07 22:04:39 alexmiller because project.clj is evaluated, can be modified by plugins, etc #2019-11-08 08:47:24 plexus @drewverlee maybe obvious but worth noting, if you control the repo you can just lein pom and check in the pom.xml next to the project.clj #2019-11-08 11:07:05 vlaaad I'm starting to feel that both lein pom and clj -Spom is a case of easy over simple. There is a dependency format — pom, that has many different stuff in it. It's much more than list of deps + artifact + version, but tools like lein pom and clj -Spom give only the most common baseline, which might cover some percentage of use cases, but whenever I want to publish a library, I realize that that's not enough. I need to specify developer/license information. I need to specify version as tag in scm information. Someone else will want to specify git sha instead of tag in scm. Dependencies I use during development might go to different profiles, with different activation rules, so what lein/`clj` provides to update dependency list might even be wrong.
I guess ideally I want to produce pom from code I write, aided with information from tools-deps/git and some helpers to generate xml from data, but ultimately be the one who decides what goes where. I guess where is no such tool yet because you can just check in pom.xml and update it with scripts using sed or whatever, and that's good enough. #2019-11-08 11:29:44 dominicm Using sed for xml was hard :) Garamond works okay in this space. #2019-11-08 11:30:25 dominicm I think it's okay to say that pom.xml is a combination of data sources into one. I personally would not want to manage the tag/version stuff as I'd definitely get it wrong. #2019-11-08 15:38:12 dominicm Yup #2019-11-08 15:38:22 dominicm Although a deps.edn is probably easier #2019-11-08 15:43:19 drewverlee what would the output of clj -Spath | grep integrant containing integrant twice imply? Im trying to undertsand a build error.
/home/drewv/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src:/home/drewv/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src/main/clojure #2019-11-08 15:45:36 alexmiller it doesn't contain integrant twice #2019-11-08 15:45:50 alexmiller it contains two source directories from within integrant (src and src/main/clojure) #2019-11-08 15:47:05 alexmiller the latter doesn't exist, so no worries #2019-11-08 15:47:27 alexmiller (that's the standard Maven source path for Clojure source files) #2019-11-08 15:49:55 drewverlee thanks alex. So {:warning "could not find classpath entry", :path "/home/drewv/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src/main/clojure"} isn't indicating something is wrong with my build? its because ____ produces that path and then a later tool can't find the entry on it? #2019-11-08 15:52:26 alexmiller I don't know what that is #2019-11-08 15:52:33 alexmiller where do you see it? #2019-11-08 15:52:56 alexmiller it's certainly not an issue to the jvm to have a classpath entry that doesn't exist #2019-11-08 15:53:28 alexmiller which I assume is why that's a warning #2019-11-08 19:24:52 seancorfield @drewverlee Integrant doesn't have a deps.edn file and that path indicates you are depending on it via a :git/url? #2019-11-08 19:25:28 drewverlee It's a fork. #2019-11-08 19:27:41 seancorfield Ah, with a pom.xml file. Got it. #2019-11-08 19:28:30 seancorfield @alexmiller Does t.d.a. assume src/main/clojure when it sees a POM file in a Git dep? #2019-11-08 19:29:07 lread hello! I am wondering, should :override-deps affect the pom.xml generated/updated by -Spom? #2019-11-08 19:29:47 seancorfield @lee If you specify an alias that brings in that override... #2019-11-08 19:30:43 seancorfield (at least, that's why I would expect -- I've never tried it) #2019-11-08 19:32:51 lread Yep, I am specifying the alias and it does not affect pom.xml in my little test. Could be I botched something. #2019-11-08 19:32:55 seancorfield Based on a quick test, it doesn't seem to. #2019-11-08 19:33:17 seancorfield (! 815)-> cat deps.edn
{:paths ["resources" "src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}}
:aliases
{:test {:extra-paths ["test"]
:extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}}}
:old {:override-deps {org.clojure/clojure {:mvn/version "1.9.0"}}}
:runner
{:extra-deps {com.cognitect/test-runner
{:git/url ""
:sha "76568540e7f40268ad2b646110f237a60295fa3c"}}
:main-opts ["-m" "cognitect.test-runner"
"-d" "test"]}}}
Fri Nov 08 11:33:00
(sean)-(jobs:0)-(~/clojure/pommy.example)
(! 816)-> clj -A:old
Clojure 1.9.0
user=> ^D
#2019-11-08 19:33:42 seancorfield But clj -A:old -Spom produces a pom.xml that uses 1.10.1 not 1.9.0 #2019-11-08 19:35:38 lread thanks sean! -Stree behaves as expected:
> clj -A:old -Stree
org.clojure/clojure 1.9.0
org.clojure/spec.alpha 0.1.143
org.clojure/core.specs.alpha 0.1.24
#2019-11-08 19:38:12 seancorfield I'd say that's a bug. @alexmiller? #2019-11-08 20:26:47 alexmiller this is trickier than it looks. I may actually have broken it in my recent set of changes. the tricky part is that the aliases are about resolving transitive deps and building a classpath, but you're not actually building a classpath here, you're just making a different manifest file #2019-11-08 20:28:02 alexmiller so if you supplied override-deps for some lib N levels down, what does that mean? do you add it to the root deps to get the same effect? or just ignore it? #2019-11-08 20:54:35 seancorfield I would certainly expect the dependencies in pom.xml to match the top-level deps in -Stree output but I agree that it's really hard to decide what the transitive dependencies should be. Hmm 🤔 #2019-11-08 21:03:54 alexmiller I used to do a bunch of fiddly logic to half apply stuff from the aliases during gen, but I stopped doing that in my last round of changes and just reused the same core from making classpaths, but did not properly take that all the way #2019-11-08 21:38:34 lread Forgive my naive question @alexmiller, I'm sure you've put much thought into this. Wouldn't adding the override of the transitive dep to the root deps give the intended result? #2019-11-08 21:39:15 lread Also do you want me to raise an issue for this? Or is it via ask clojure now? #2019-11-08 21:50:31 alexmiller I've actually fixed it locally, so no need to do anything. I'll probably log a jira just for tracking before I commit it #2019-11-08 21:51:09 alexmiller on the transitive deps, the decision I had made in the past was to only modify the root deps and not include overrides in transitive deps #2019-11-08 21:52:49 alexmiller and that's what I'm matching in the update. inevitably, all decisions here will be wrong for someone. #2019-11-08 21:55:37 lread true enough! So long as behavior is consistent and easy to understand, it works for me! (for what that's worth!) #2019-11-08 22:16:42 lread cool, thanks alex! #2019-11-09 22:05:45 sogaiu somehow i am reminded of boot and badigeon #2019-11-09 22:06:13 sogaiu @dominicm thanks, had never seen that 🙂 #2019-11-09 23:26:36 alexmiller Lancet was the long running example in the first edition of Programming Clojure #2019-11-12 20:03:01 alexmiller I'm going to do a talk about tools.deps / clj next week at the conj - any burning questions I should be sure to answer? #2019-11-12 20:06:04 dominicm Settle the debate on namespaces. Talk about java. Leak the plans for tools.ant #2019-11-12 20:07:45 seancorfield @alexmiller Talk about add-lib 🙂 #2019-11-12 20:10:48 alexmiller @dominicm if those are serious, I'm not sure what any of them mean #2019-11-12 20:12:40 dominicm I'm just teasing :) I was hoping tools.ant would give it away.
Unless there really is a tools.ant... #2019-11-12 20:13:12 alexmiller well, Lancet was kind of like tools.ant :) #2019-11-12 20:16:19 dominicm It would be interesting to know if there's ideas you have that you'd like others to execute. (E.g. Expanding on Lancet, building X, the community would gladly step up and at the least explore the domain for you) #2019-11-12 20:19:58 alexmiller yeah, I'll cover some future idea stuff for sure #2019-11-12 20:11:00 alexmiller @seancorfield maybe, we'll see #2019-11-12 20:11:45 dominicm In earnest, the big questions I've had are:
- how do I do java
- how do I do thing I do with lein (usually uberjar)
- how do I load the tests of my dependencies (usually in the context of a monorepo)
- why would I use this rather than lein #2019-11-12 20:12:29 alexmiller "do java" == "compile java"? #2019-11-12 20:13:19 dominicm Sorry, yes. How do I fold java into my program, or use java libraries from source. Stuff like that. #2019-11-12 20:13:28 alexmiller don't? :) #2019-11-12 20:14:15 dominicm Good slide, yep :) (Although the team I said that to then introduced lein and took away all my lovely deps.edn) #2019-11-12 20:17:40 seancorfield Not so much "do java" but "do compile" in the presence of gen-class etc I think would be interesting to at least mention (`compile`, binding the output directory, making sure it exists). #2019-11-12 20:18:28 alexmiller you guys are too far on one end of the audience distribution :) #2019-11-12 20:18:28 dominicm These are really clojure questions, but is interesting how lein did all this stuff that we didn't understand. Even though it's simple to do yourself. #2019-11-12 20:19:20 alexmiller but I will keep those in mind #2019-11-12 20:20:08 dominicm I guess the recurring theme is "lein did X, deps.edn doesn't, where do I go?" #2019-11-12 20:21:59 seancorfield https://github.com/seancorfield/cfml-interop/blob/master/deps.edn -- does compile of a gen-class namespace (caveat about the compile target dir needs to exist). I don't think compile/AOT is that far too one end of the spectrum really -- lots of people seem to assume you need AOT because "everything" out there assumes :aot main.core and uberjar with a compiled -main 🙂 That's a big difference when you switch to CLI/`deps.edn` from the lein world (or even the boot world). #2019-11-12 20:26:28 seancorfield Intro to lein tends to be lein new app myapp; lein run; lein test; lein uberjar; java -jar path/to/the.jar -- so I think the equivalent story with CLI/`deps.edn` needs to be out there "officially" with at least some explanation of the differences. #2019-11-12 20:27:09 seancorfield (the current docs expect you to create your need project completely manually, for example, right?) #2019-11-12 20:36:16 alexmiller where "build" == what? #2019-11-12 20:44:29 seancorfield @alexmiller edited. Sorry, meant "create". #2019-11-12 20:36:36 alexmiller create and deploy an artifact? #2019-11-12 20:37:24 alexmiller "build" by itself is not a very precise word in this context #2019-11-12 20:37:50 vlaaad how to build your own lein with tools-deps 🙂 #2019-11-12 20:38:07 vlaaad when TDEPS-116 will be solved 😄 #2019-11-12 20:39:35 alexmiller it'd be a lot better if that ticket talked about a problem, not a solution #2019-11-12 20:39:47 vlaaad it is #2019-11-12 20:40:08 alexmiller well, not by my reading #2019-11-12 20:41:10 vlaaad ...just not very clearly: I want to author more fine-grained libraries that may have optional code/dependencies which won't bloat classpath when unneeded. With maven it can be done using classifiers. With gitlibs, it's all or nothing #2019-11-12 20:42:52 alexmiller I have like 3 questions about every part of that statement #2019-11-12 20:43:20 vlaaad I'm ready to answer! #2019-11-12 20:43:42 alexmiller but I'd suggest pulling the "optional code/dependencies" string as a place to start #2019-11-12 20:45:19 vlaaad I don't have an account there unfortunately, can't edit the issue. #2019-11-12 20:45:20 alexmiller I don't have time to have a long conversation about this right now but something like that is a far better title/thing to describe in a ticket. #2019-11-12 20:46:05 alexmiller "Allow specifying aliases in coordinates that point to deps.edn projects" is a specific change that might solve a problem, but unlikely to be the only possible change to solve those problems #2019-11-12 20:48:24 alexmiller Maven classifiers by themselves are just another dimension to your artifact coordinate (and tdeps already supports classifiers). They only get at this kind of a problem in combination with the Maven property system that allows you to conditionalize a dependency coordinate. #2019-11-12 21:12:19 sogaiu may be it's not worth saying to this audience, but does the following seem off?
my general impression was that clj / deps.edn tries to address some key issues (e.g. pomegranate-related, git deps, etc.) and wasn't out to replace lein, boot, etc. i thought a somewhat natural path would be for existing tooling to start using clj / deps.edn (or tools.deps, gitlibs, etc.) to take advantage of its capabilities. iiuc, there are also new efforts to build on or make use of tools.deps + related: https://github.com/juxt/pack.alpha, https://github.com/EwenG/badigeon, etc.
i've used at least one lein plugin (https://github.com/reifyhealth/lein-git-down) which, iiuc, makes use of gitlibs with varying degrees of success -- and iiuc there are other plugins (see the bottom of lein-git-down's README for some) that have made attempts. may be that these or similar things haven't "made it into the core of lein" is where some pain comes from? #2019-11-12 21:23:29 andy.fingerhut If your future work part drops any hints about the more recent version of codeq tht you mentioned a few weeks ago in chat, you might be inundated with dozens of requests for more info, unless it is provided during your talk 🙂 #2019-11-12 21:39:40 alexmiller that has nothing to do with tools.deps so dont see why it would come up #2019-11-12 21:30:58 seancorfield @sogaiu I went down that path with boot-tools-deps but ultimately it was too difficult to meld t.d.a. and Boot due to their different worldviews on paths and dependencies for anything but the very simplest use cases. I don't know how lein-tools-deps fares in that regard. #2019-11-12 21:32:00 seancorfield I think if lein/`boot` adopted t.d.a. wholesale as their way to handle paths and dependencies at their core, then that "natural path" could happen -- but that's a lot of rework and potentially breaking as well. #2019-11-12 21:35:05 seancorfield And it's turned out that the effort that might have gone into that has perhaps gone into building tools on top of t.d.a. that provide the Leiningen-like features as point solutions you can mix'n'match (which is, to some extent, Clojure's general mindset). Both Leiningen and Boot are "easy" rather than "simple" because they are both "batteries included" -- which have served the community well for a long time but the CLI and the t.d.a.-based tools we have now feel more "Clojure-y" to me. #2019-11-12 21:41:11 sogaiu thanks for the insights #2019-11-12 21:45:42 vlaaad all "burning questions" for Alex to answer seem to be answered immetiately 🙂 #2019-11-12 21:48:54 vlaaad I agree wholeheartedly with Sean, tools-deps are simple and that empowers me to quickly hack together tools I personally need, because I don't have to deal with semantics imposed by build tool #2019-11-12 21:50:40 vlaaad I think there is a lot of space for libraries to emerge that help with that process of developing projects #2019-11-12 21:54:40 vlaaad I made a couple of libraries with tools-deps, and the thing that wasn't there is aid to generate pom.xml, for example. clj -Spom is not enough — it's a good starting point, but my build scripts usually have to update different fields in pom, and dependency list provided by tools-deps might end up in different place... I'd like to see a tool that just helps me generating pom from data, and then I will use tools-deps programmatic APIs to provide information I need #2019-11-12 21:58:23 seancorfield Yeah, having a minimal pom.xml that satisfies the needs of clojars/cljdoc would be a big improvement over what -Spom produces by default but that would be a fairly simple tool to write I guess. Also something to update the version/tag in the POM file. #2019-11-12 21:59:15 seancorfield (shouldn't be core t.d.a./CLI functionality -- the community can easily build this... I just haven't quite felt enough pain yet to do write it myself 🙂 ) #2019-11-12 22:00:25 seancorfield I could probably even build that into clj-new to be honest so at least you start with a somewhat fleshed out pom.xml file...? #2019-11-12 22:01:38 kszabo > Pom.xml generation
>I have added the option to generate a pom.xml file based on what is already available from clojure.tools.deps. It actually uses the sync-pom function from tools deps then goes to the generated/updated pom and adds the group-id, artefact-name and version based on what metav already establishes. #2019-11-12 22:01:46 kszabo this looks like it fits the bill #2019-11-12 22:04:12 kszabo we are very happy users of metav so this is a welcome addition, maybe it would be nice to add this dependency to clj-new? We are mainly using it’s sha-based versioning but of course folks could use it for semver (who haven’t watched Spec-ulation 🙂 ) #2019-11-12 22:06:51 alexmiller I haven't looked at it in many years, but Maven actually has support for storing pom data in arbitrary formats and there was even a clojure format way back when this was conceived of #2019-11-12 22:08:25 alexmiller maybe that's just taking on a lot more problems :) #2019-11-12 22:10:15 dominicm Is there a dimension in which maven is not extensible? #2019-11-12 22:10:59 alexmiller sadly, no #2019-11-13 18:42:53 rickmoynihan How do people manage and maintain their aliases?
For example in one of our projects:
To start a basic dev repl (with all configured customers data on the classpath) you need the aliases: -A:dev:test:cljs:prj/all
For me to start a dev repl I need/want to add the extra aliases :cider:sc to those.
To run the tests in CI or via kaocha you need: -A:kaocha:test:prj/all:cljs
To build a specific release for a specific customer we need: -A:prj/customer-1:prod
Plus there are various other configurations of aliases. Right now I tend to “document” these by having scripts e.g. ./bin/cider ./bin/repl ./bin/kaocha apply them. However this is opaque to the tooling.
So cursive users have to manually select the specific aliases they want to translate the scripts to the aliases they want to use when they start a repl; and Emacs users need to run the right script with their own :cider alias appended and cider-connect to the repl.
Then a different project will have different combinations.
I find our alias definitions tend to become smaller over time, so they can be reused in other contexts — but then you end up with lots of small things that you have to remember to add back in when you start your REPL/tests/etc.
It’d be nice if tools deps aliases could also be sets of aliases… e.g. I’d really like to define the following:
:composite-aliases {:dev #{:prj/all :test :cljs :user/dev}
:prj/all #{:prj/customer-1 :prj/customer-2 :prj/customer-3 ,,,}
:prod/customer-1 #{:prod :prj/customer-1}
:prod/customer-2 #{:prod :prj/customer-2}}
#2019-11-13 19:25:53 seancorfield Alias-aliases like that would be very handy sometimes. #2019-11-13 19:34:03 hiredman At work @seancorfield wrote a shell script that manages a lot of that stuff for us. Outside of work I rely pretty heavily on shell history for that kind of thing (type it in once, that search for it in history to run it again forever after), which has the added benefit of working for things outside of clojure too, but is not really sharable for a team. #2019-11-13 19:46:33 seancorfield That script knows about the high-level aliases and expands them into the set of low-level ones. It also knows how to run multiple clojure commands in sequence, to simulate lein/boot's ability to run multiple commands in one go #2019-11-13 20:03:45 hiredman I think you might be better off creating coarser aliases. It seems to often be the case then when you add a meta language to a system (in this case aliases in deps are an aggregate built on top of clojure functions, but you can draw a similar analogy with things like di systems) the meta language starts out providing a coarse aggregate of base level functionality, and then overtime becomes finer overtime until it corresponds so closely with the base level that you lose the aggregate functionality and start building a new meta level (:composite-aliases) to try and regain the aggregation #2019-11-14 09:18:30 rickmoynihan > I think you might be better off creating coarser aliases
Yeah, I’ve tried that too. That’s essentially what the :prj/all alias is. It’s an alias that adds all (customer) project dependencies to the classpath; and it is essentially the union of all the customer aliases.
I find it becomes hard to then know what deps/paths/config belong to which alias though… e.g. I’m a believer that in the repl you should have both :dev-tooling and :test stuff available. But :test may also want to be included in the alias the CI uses to run the tests.
The problem then is that the groupings become lost, and you need to resort to comments to group things which is far from ideal. #2019-11-13 20:05:41 hiredman like, maybe get rid of the separate alias for adding cider, add it to the class path for all the dev aliases, but add some kind of runtime configuration option so it only launches nrepl on developers machines that want it #2019-11-13 20:08:52 dominicm @rickmoynihan 1. copy and paste 2. Not including things like cider in projects, that's userland stuff #2019-11-14 09:21:49 rickmoynihan I agree cider is userland stuff. We treat it as such.
This adds a further complication that each user might define cider in a different alias, e.g. I have :cider-22 :cider-21. #2019-11-13 20:10:22 dominicm You can probably configure cider to call clojure always as -A:cider <prompted arguments>. #2019-11-14 09:28:17 rickmoynihan Yeah I think you can do that but often we need to run the JVM in a particular environment, with env vars etc set…
So I tend to prefer to cider-connect to an instance started in a terminal.
I’ve solved some of the issues here by starting the dev repl and dependent services, along with shadow-cljs --watch etc with overmind; which uses a Procfile to start stuff; and allows you to share env vars across services.
I then use direnv to share project wide, not just repl wide, configuration… e.g. a port/location of a shared database across services etc. #2019-11-13 20:11:13 dominicm Dev/test/cljs would probably be copy and pasted into each other as much as they were mutually dependent. #2019-11-13 20:11:33 dominicm I'm not sure what's different between customers for you, so it's hard to say what I'd do #2019-11-14 09:30:33 rickmoynihan Mainly just assets on the resource path; and occasionally customer specific reference data. #2019-11-14 09:34:14 rickmoynihan Though some of those assets are bundled into webjars… e.g. css skins. #2019-11-14 09:46:23 plexus I quite like having wrapper scripts for this, this is also why I promote adding a bin/kaocha to projects. That also gives you an opportunity to further reduce friction by doing extra setup steps, doing precondition checks (e.g. minimum java version), setting environment variables, passing along JVM options, ... #2019-11-14 10:00:03 rickmoynihan @plexus: Yeah we use kaocha; it’s great, thanks for that btw 🙂
Though I think adding bin scripts is a bit of a hack/workaround, as it’s opaque to introspection and tool support. Which is another thing projects (particularly internal ones) then need to document; often for each editor/environment in use on the team. #2019-11-14 10:57:53 plexus I agree, but I think that's exactly why they are great, they are executable documentation. I'd rather see bin/cider_repl / bin/proto_repl / ... then having one liners in the README (not too bad) or having them shared ad-hoc through back channels (more common). It removes friction and cognitive overhead for the most common cases. You can still pass on extra args to clojure, and if that doesn't cut it then run it with bash -x, copy the final command into your shell and take it from there. #2019-11-14 10:59:14 plexus it also means most developers are doing the same thing. I've been on too many projects where it turns out every single person runs their dev process slightly different from everyone else, causing all kinds of reproducibility issues. #2019-11-14 11:33:24 rickmoynihan TIL. nice tip:
> run it with bash -x, copy the final command into your shell and take it from there. #2019-11-14 11:36:04 rickmoynihan > I’ve been on too many projects where it turns out every single person runs their dev process slightly different from everyone else, causing all kinds of reproducibility issues.
I think this is a side effect tools.deps has encouraged, by doing less.
Expectations and workflows on lein projects were typically more uniform. #2019-11-14 17:51:32 seancorfield It's why we have a standardized build script that runs clojure, and also why we have a standardized "baseline" deps.edn which we control via CLJ_CONFIG in the build script. #2019-11-14 17:52:18 seancorfield No user-level aliases interfering. No possibility of different versions across the team. #2019-11-14 17:52:41 seancorfield The baseline deps.edn contains all the aliases that anyone needs. #2019-11-15 10:54:23 rickmoynihan So assuming I was on your team for a second…
Presumably I’d need any personal aliases to be committed into that file? What about if I wanted to use classpath overrides etc, to stub a dep with a localised version etc? I like to use scope-capture, assuming noone did already would that need to be a conversation with the team, or would I just add it in as a dep into everyones :dev alias?
I’m curious how the build script works… Does it start a repl as well as build stuff? Does it accept aliases as arguments? Or does it manages alias groups? i.e. ./build repl cider would start it with cider ./build repl cursive suitable for cursive etc?
What environments do people on your team use? Does everyone use chlorine and nrebl like you? Or do folk also user cider, intellij etc? #2019-11-15 10:57:40 rickmoynihan You have me thinking though, that there’s a benefit to the consistency wrt to transitive deps of always at least having things like the cider deps on the classpath even if not everyone uses it; it wouldn’t need to be loaded after all 🤔 #2019-11-15 11:13:40 danieroux We optimise for pairing. And therefore value consistency very high.
We have Makefiles. And a make repl creates an nRepl on a known port, and both Cider and Cursive is configured to connect to it. We also share configurations. #2019-11-15 11:41:48 dominicm Cider doesn't really have deps #2019-11-15 13:18:42 rickmoynihan yeah they’re shaded by mr anderson — but not all tools/libs will do this. #2019-11-15 14:27:43 dominicm Then we must be careful in our selection #2019-11-15 14:28:13 dominicm Pick development tooling that itself only has safe deps #2019-11-15 14:29:31 rickmoynihan if its always there yeah… if it’s something you can opt in to it doesn’t matter as much #2019-11-15 14:32:34 dominicm I would say it's the other way round 😊 if it is bundled with the project and is always present then it doesn't matter. If I load it as a user, then it ought to be stable so I know I'm not causing the reproducibility problems that were mentioned. #2019-11-15 16:45:25 dominicm That's a good example of something you only use sometimes #2019-11-15 11:42:12 dominicm Tooling dependencies have to be carefully selected because of what you describe #2019-11-15 11:43:14 dominicm As someone who likes to tinker and improve on my development process, I find the idea of not supporting personal aliases distasteful. #2019-11-15 16:54:47 seancorfield No one on our team uses CIDER or nREPL these days but there are still two aliases in our shared deps file for starting such things (old and new versions). #2019-11-15 16:58:21 seancorfield Yes, the build script takes a list of aliases to use. Yes, it can start a REPL, REBL, Socket REPL, and several other variants (including old and new nREPL servers). #2019-11-18 15:26:57 practicalli What does the S in -S option for the Clojure CLI tools stand for? The other option names seem to be fairly obvious in meaning when reading clojure --help. For example -A is aliases, -C is classpath, -R resolve, etc.
Or is -S just a generic 'catch all' prefix for all other commands? #2019-11-18 15:46:12 dharrigan If I was to hazzard a guess, I think it means script - as examining the clojure script file that controls the process, S is used for script commands (internal to the script) and also a tree function, in the script package in tools deps alpha. #2019-11-18 15:46:23 dharrigan I could be 100% waaaay of the mark here ... 🙂 #2019-11-18 16:09:26 alexmiller It stands for "settings" #2019-11-18 16:09:56 dharrigan aha! #2019-11-18 16:09:57 dharrigan 🙂 #2019-11-18 16:09:57 alexmiller so, yes a generic prefix to catch many settings #2019-11-18 16:11:05 alexmiller in particular, the command line has a mix of clj args, clojure.main args, and when used as a runner, your own clojure main args #2019-11-18 16:31:04 sogaiu at least now i have two mnemonics :) #2019-11-18 18:26:03 practicalli my brain can rest at ease now, thank you Alex. #2019-11-18 23:32:42 danielsz @alexmiller Hi Alex, the console-listener Var in clojure.tools.deps.alpha.util.maven appears to be an implementation of the TransferListener interface. However it isn't a complete implementation, and I am seeing errors to that effect. #2019-11-18 23:32:51 danielsz Receiver class clojure.tools.deps.alpha.util.maven$reify__7307 does not define or inherit an implementation of the resolved method 'abstract void transferSucceeded(org.eclipse.aether.transfer.TransferEvent)' of interface org.eclipse.aether.transfer.TransferListener. #2019-11-18 23:34:59 danielsz Please note that the docs suggest to extend the abstract class AbstractTransferListener rather than implementing the interface. The abstract class ships with no-op callbacks so a partial extension is a-OK and no errors would occur. #2019-11-18 23:41:45 alexmiller Thx #2019-11-20 20:00:33 dadair What's the recommended way for evaluating the dependency tree for deps.edn based projects (e.g., lein deps :tree)? I've tried clj -A:dev:test -Spom && mvn dependency:tree -Dverbose=true, but I just get an exception that maven can't resolve the project deps #2019-11-20 20:01:00 dadair I'm trying to find out if a dep somewhere is loading an old tools.reader that ends up spitting out some warnings in my project #2019-11-20 20:02:23 dadair I've used clj -A:dev:test -Stree, but wasn't sure if there is a more verbose one, as the -Stree approach only shows one dependency on tools.reader and it's definitely of a version that includes a fix to not have the warnings.. #2019-11-20 20:05:34 nwjsmith @dadair do you happen to have cloverage in your test dependencies? #2019-11-20 20:05:35 alexmiller can you share the exception? #2019-11-20 20:07:02 dadair @nwjsmith yes, through kaocha-cloverage #2019-11-20 20:07:33 nwjsmith it contains an AOT-ed version of tools.reader, which doesn’t show up in the dependency tree #2019-11-20 20:08:15 alexmiller evil #2019-11-20 20:37:54 dadair Cannot target the sha via deps as cloverage doesn't provide a deps.edn #2019-11-20 20:09:05 nwjsmith but a release needs to be cut for it (only @lvh can do that) #2019-11-20 20:09:27 nwjsmith and we should probably stop AOT-ing that dependency #2019-11-20 20:09:32 nwjsmith I’ve had problems with it too #2019-11-20 20:09:33 dadair @alexmiller Could be of note, I am running through WSL2 (Ubuntu 18.04) #2019-11-20 20:10:50 alexmiller looks like missing clojars repo or something #2019-11-20 20:14:33 lvh Hi: I think I mentioned this in a ticket somewhere but CI should be building snapshots #2019-11-20 20:14:45 lvh I’m on my way to the Conj right now so I can’t check #2019-11-20 20:17:52 nwjsmith Oh! good call. @dadair you might get rid of those warnings by adding cloverage {:mvn/version "1.1.3-SNAPSHOT"} to your deps.edn. #2019-11-20 20:20:16 dadair Doesn't seem to be making a difference #2019-11-20 20:21:51 dadair Confirmed that the override is in place, just not working
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "412520372825010504120a150e116c050c7200091200"}, :content ("[email protected]")} #2019-11-20 20:24:59 alexmiller If cloverage has the other one and ends up first in the classpath it will win #2019-11-20 20:25:20 alexmiller Whether it’s first is pretty arbitrary #2019-11-20 20:26:04 alexmiller -Spath should be exactly what you’re running #2019-11-20 20:29:59 dadair hmm, using -Spath, cloverage is on the snapshot version (1.1.3-SNAPSHOT), and tools.reader is at the fixed version 1.3.2, yet the warnings are still printing.. #2019-11-20 20:37:47 miikka FWIW, cloverage-1.1.3-20191022.214104-8.jar still contains AOT'd tools.reader #2019-11-20 20:39:46 miikka Oh, right, as it should #2019-11-20 22:12:58 alexmiller you can take clj out of the loop by doing java -cp <the-path-from-Spath> clojure.main <your-main-args> and then feel free to reorder / edit the cp to try things #2019-11-21 06:33:04 miikka The Cloverage snapshot still seem to cause the problem. I get the warning with the following classpath by running java -cp ... clojure.main -e "(require 'clojure.tools.reader)":
/Users/miikka/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/miikka/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/miikka/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/miikka/.m2/repository/cloverage/cloverage/1.1.3-SNAPSHOT/cloverage-1.1.3-SNAPSHOT.jar
#2019-11-21 06:52:46 miikka I'm now wondering if :refer-clojure :exclude and AOT actually work correctly together. #2019-11-21 06:53:11 miikka Should be easy to test, but I have to do something else now. #2019-11-21 14:22:45 alexmiller clj 1.10.1.489, tools.deps.alpha 0.8.599
* TDEPS-143 - Fix: use aliases when generating pom with -Spom
* TDEPS-127 - Add: include pom resource directories in dep paths in -Spom (thanks Miikka Koskinen)
* TDEPS-140 - Add: support for Maven server HTTP header properties (thanks Arne Brasseur)
* Fix: Use of multiple aliases in Windows clj script
* Add -Strace for debugging dep expansion #2019-11-21 19:06:27 mss getting a weird error when attempting to use depstar to build an uberjar:
Execution error (FileAlreadyExistsException) at com.sun.nio.zipfs.ZipFileSystem/createDirectory (ZipFileSystem.java:422).
META-INF/versions/9/
#2019-11-21 19:07:46 mss I can’t tell if I’ve configured something incorrectly in intellij, if I’m missing an option in building the classpath, or something else entirely. not much is coming up when googling around about a META-INF related FileAlreadyExistsException #2019-11-21 19:09:11 mss if people have a recommendation for tooling to use to build standalone jars, I’m all ears as well #2019-11-21 19:10:11 mss interestingly, using depstar to build a slim jar and attempting to run that jar results in a Error: Invalid or corrupt jarfile error #2019-11-21 19:39:31 seancorfield @mss I'm at Conj so I won't be able to look at that until next week but if you can put a repro case up on github and create an issue against depstar with a link to your repro case, that would really help #2019-11-21 19:41:01 seancorfield You can't run a "slim jar" - you can only use that as a library (since it won't contain any dependencies). #2019-11-21 21:40:30 jfntn Is there a supported approach for merging deps from a parent directory in a monorepo? #2019-11-21 23:09:03 seancorfield @jfntn At work we use the CLJ_CONFIG environment variable to point to the "master" deps file so that it gets combined with the subproject's deps file within our monorepo. #2019-11-21 23:10:19 seancorfield cd subproject
CLJ_CONFIG=.. clojure
(our master deps file is in ../versions not .. but the same principle applies) #2019-11-22 18:09:28 jfntn Interesting but that would prevent using user defined aliases etc right? #2019-11-22 18:10:21 jfntn Also how do you enforce the env var, do you have to use a makefile or some additional env tooling? #2019-11-22 23:02:04 seancorfield @jfntn Yes, a build shell script that runs clojure one or more times so we can have a single command run multiple tasks, all with CLJ_CONFIG set. And, yes, no user-defined aliases can interfere with our build process. Any developer tooling is available via aliases in that master deps.edn file. #2019-11-23 12:20:24 daveliepmann I borked a copy/paste in my deps.edn and briefly had a dependency pointing to a different git repo. In other words the key is library X but the git repo coordinate is library Y. What's the expected behavior in this situation?
org.clojure/tools.deps.alpha {:git/url ""
:sha "e5f3d9e6839191f1e37ddfa51cf442b2d5403ff3"} #2019-11-23 12:33:50 alexmiller There’s no way for tda to tell #2019-11-23 14:18:42 kszabo I am getting:
Error building classpath. Could not find artifact org.clojure:tools.deps.graph:jar:0.1.5 in central ()
#2019-11-23 14:23:02 dominicm The repo was empty a moment ago #2019-11-23 14:25:01 seancorfield Maven Central can take several hours to list a new release. I suspect that this is something Alex will debut in his talk in a few hours. #2019-11-23 14:27:32 kszabo I wish I could be there 😢 #2019-11-23 14:29:16 seancorfield All the talks are showing up on YouTube already! #2019-11-23 14:36:01 alexmiller that was a dummy version - it's up there now at 0.1.14 #2019-11-23 14:49:14 dominicm The conj is still on?! Yay, more to watch #2019-11-23 14:49:29 dominicm Having the videos go up live is fantastic for those of us across the pond #2019-11-23 15:48:46 sogaiu even on the same lilypad, it's much appreciated 🙂 #2019-11-23 15:54:12 seancorfield Even if you are actually here it's great. I missed a talk yesterday but was able to watch it online later before going out for dinner! #2019-11-23 16:07:05 sogaiu nice! #2019-11-23 14:32:46 alexmiller shhh #2019-11-24 15:29:33 andy.fingerhut Alex, you mentioned what I think you called logically inconsistent deps that one occasionally comes across in the deps of projects. Is there an existing option to clj/clojure, tools.deps, or some other tool that tells one if a project has such logically inconsistent deps? #2019-11-24 15:36:28 alexmiller No, it will just muddle through #2019-11-24 15:49:26 andy.fingerhut Sure. Just wondering for yours (and others) debug purposes if something like that existed, as a machine-detectable property that can signify that here might be dragons. #2019-11-24 20:26:29 lvh I want to run a tool specified in a deps.edn file as an alias (i.e. with a :main-opts), but with a multiple sets of command line arguments to debug an issue down to a particular combination of arguments. do I just use clojure.main repeatedly or is there a smarter way to do that? #2019-11-26 13:41:03 unbalanced Good morning! Darn conj has me so full of ideas now 😮
So what does it take to hook into the tools-deps machinery? Is there a general primer-faq? Reason I'm asking is b/c I'm working with Chris Nuernberger on libpython-clj and I think it would be a very good idea to be able to specify Python deps declaratively. Is tools-deps designed to be extensible like that or should we use a seperate .edn file inspired by deps.edn? #2019-11-26 13:42:09 dominicm It is extensible, but the clj tool currently doesn't define a way for extensions to be loaded #2019-11-26 13:43:42 vlaaad I thinks it's not extensive in that direction. it's a tool to resolve java dependency and build java classpath, it probably can't be extended for other ecosystems such as python or js. they are too different. #2019-11-26 13:49:27 alexmiller clj is about making classpaths so the question is whether there is any use in putting python stuff on a classpath #2019-11-26 13:51:42 unbalanced So the use case here is for library authors using the libpython-clj tool #2019-11-26 13:51:54 alexmiller You could use just the resolve-deps part with custom providers to do the transitive expansion #2019-11-26 13:52:28 unbalanced There's a chicken/egg problem -- basically the Python VM needs to be loaded before any libpython-clj libraries can be pulled in #2019-11-26 13:52:32 alexmiller Unclear if that’s worth doing via tolls deps #2019-11-26 13:53:13 unbalanced Of course! I was't sure if it was designed for that or not #2019-11-26 13:57:51 alexmiller I don’t know enough about python stuff to tell but the provider extensions are pretty generic so it could probably be made to work, just not sure if it’s worth doing #2019-11-26 14:40:44 orestis Python has literally a PYTHONPATH which is pretty similar to the classpath. But in practice I think people don’t do that. The Python community changed the “de-facto” dependency manager 4 times in the last 5 years already (I haven’t looked deeply into python rencetly, might be wrong). #2019-11-26 15:35:56 dominicm I think they still use the same repo though don't they? #2019-11-26 15:37:07 dominicm The real question is:
- How does libpython-clj search for python code? It could search the classpath, jruby can.
- How does graal python find python code?
Those are the two consumers I can think of. #2019-11-28 17:55:19 souenzzo How do I import https://mvnrepository.com/artifact/org.apache.cxf/cxf/3.3.4 in deps.edn?
I'm trying org.apache.cxf/cxf {:mvn/version "3.3.4" :extension "pom"}
It add a .pom to my classpath, but I can't import anything
(filter
#(string/includes? % "cxf")
(string/split(System/getProperty "java.class.path") #":"))
=> ("/home/souenzzo/.m2/repository/org/apache/cxf/cxf/3.3.4/cxf-3.3.4.pom")
#2019-11-28 18:06:04 souenzzo Okay. So I will not be able to use this dep at this time? #2019-11-28 18:06:42 seancorfield You'll need to manually specify all the deps that the POM specifies I think. #2019-11-28 18:07:14 seancorfield I thought there was a way for t.d.a. to pull in things via a pom dep but I can't think what it is right now... #2019-11-28 18:14:15 dominicm You definitely don't want it on the classpath, that's for sure 🙂 #2019-11-28 18:16:11 souenzzo Nice to know #2019-11-28 22:36:53 sogaiu i am seeing what looks like extra strings "/main" and "/java" being tacked on to a couple of the non-jar items of a classpath stored in some files in .cpcache.
specifically, for: https://github.com/FundingCircle/fc4-framework/tree/master/tool#running-the-tests , running clj leads to .cp and .libs files containing:
• .gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/a1b1b0903ae5ba56eaada8496ed3785a53d8cc06/src/main
• .gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/a1b1b0903ae5ba56eaada8496ed3785a53d8cc06/src/java
looking at the repository for the library in question (https://github.com/tatut/clj-chrome-devtools), i see a src directory but no subdirectories named "main" nor "java".
there is a deps.edn file, but it has no :paths key
is this expected behavior? #2019-11-28 22:43:22 seancorfield The :paths in fc4-framework/tool/deps.edn has src/main so that will propagate I believe. Not sure where src/java is coming from. What's in your ~/.clojure/deps.edn file? #2019-11-28 22:46:39 seancorfield Hmm, perhaps not. I just cloned the repo and ran clj -A:dev:test in the tool folder and when I look in .cp I do not see src/main or src/java #2019-11-28 22:46:59 seancorfield (! 858)-> cat .cpcache/1648314215.cp | tr ':' '\n'|fgrep gitlib
/Users/sean/.gitlibs/libs/expound/expound/297e0812eb77ae3aa8f30e823f8bdbbb4650fa28/src
/Users/sean/.gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/a1b1b0903ae5ba56eaada8496ed3785a53d8cc06/src
#2019-11-28 22:47:20 seancorfield ^ @sogaiu Maybe this is from your user-level deps.edn file? #2019-11-28 22:53:06 sogaiu @seancorfield thanks for checking
my ~/.clojure/deps.edn file is basically a slightly modified version of your public one 🙂 grepping for java in it doesn't turn up any culprits (just the javafx deps for rebl)
i see my clojure-tools is 1.10.1.447 though, may be i should update that and see if it makes any difference #2019-11-28 22:57:57 seancorfield (! 861)-> clj -Sdescribe
{:version "1.10.1.492"
:config-files ["/usr/local/Cellar/clojure/1.10.1.492/deps.edn" "/Users/sean/.clojure/deps.edn" "deps.edn" ]
#2019-11-28 22:59:20 seancorfield (after updating your CLI, remember to delete .cpcache to make sure it regenerates it) #2019-11-28 23:02:21 sogaiu thanks -- what you said about propagating seems to be happening. i failed to mention i was not on the master branch. the branch i'm on has "src/java" as one of its paths. #2019-11-28 23:03:12 seancorfield Ah, interesting. With the latest CLI I see src/main in .cp but only for the tool repo itself, not for any transitive deps. #2019-11-28 23:03:43 sogaiu fwiw, the branch i'm trying is native-image #2019-11-28 23:03:43 seancorfield So if you update and re-generate and no longer see src/main/`src/java` then it sounds like a bug that got fixed since .447 #2019-11-28 23:03:59 sogaiu right -- will work on that #2019-11-28 23:05:21 seancorfield I switched to that branch, deleted .cpcache, and re-ran clj -A:dev:test and now I see src/java in .cp but only for the and it does not propagate into the git deps. #2019-11-28 23:10:02 sogaiu thanks for checking -- hopefully upgrading will fix it 🙂 #2019-11-28 23:16:10 sogaiu @seancorfield upgrading seems to have fixed things -- thanks a lot for your help 🙂 #2019-11-30 20:40:23 Nolan tools.deps.graph is wonderful. the missing piece of rooting out any unexpectedness in deps projects. 🙏 x`Inf` @alexmiller #2019-12-02 04:26:34 mike_ananev Hi! Is there any way to build WAR using tools deps? I know only leiningen plugin for building war, but it is not convenient to use both tools. #2019-12-02 04:58:47 seancorfield But a WAR is "only" a JAR/ZIP with some additional metadata files in it... #2019-12-02 08:15:20 dominicm Someone requested it for pack, but I never got round to it because I'm not very familiar with war. I think it should be straightforward using the internal API, because the layout is very similar to the other supported types. #2019-12-02 21:26:34 didibus How does tools.deps resolve conflicts for git deps and local deps ? #2019-12-02 21:36:17 seancorfield I believe it can't resolve conflicts across different types of deps? For two git deps it can resolve to the "most recent SHA" I think. Not sure how it would handle two conflicting local deps. #2019-12-02 23:09:58 alexmiller Git - newest wins, local - nothing implemented now but I think that is rare #2019-12-02 21:41:56 seancorfield I can't find a JIRA issue for this. @alexmiller is there an issue for this that I'm just not seeing? Or is this one of those "known issues" that hasn't gotten as far as JIRA yet? #2019-12-02 23:10:49 alexmiller There is a ticket similar to this but I could not repro it last week, hoping to clarify soon #2019-12-02 21:43:12 seancorfield @didibus It feels like a local dep should always be preferred over a git dep which in turn should be preferred over a Maven/Clojars dep but maybe that's just my misguided intuition... #2019-12-02 22:17:07 didibus Ya, it can be tricky. I wonder if introducing a deps concept of version could help, like if you declared the version in deps.edn #2019-12-02 22:17:40 didibus Now it could be used to compare across revolvers #2019-12-02 22:18:49 didibus Another thing could be to allow to specify a version when the resolver has no concept of it. So I declare a local deps, maybe I could also tell tools.deps what version it is #2019-12-02 22:19:13 didibus But that one would seem more confusing #2019-12-02 23:06:34 alexmiller Yeah, not planning to do that. Most Maven artifacts include their shas (central enforces, not sure if Clojars does) which gives you comparison basis w git deps (even across forks) #2019-12-02 23:08:12 alexmiller Not all cases will be resolvable - in that case it’s an error and you (human) must resolve by picking one via top dep or override #2019-12-02 23:27:09 seancorfield @alexmiller Does that mean that if you have a :git/url and a :mvn/version and the pom.xml for the latter includes a <tag> that t.d.a. currently resolves that to a SHA and figures out whether it's before/after the :git/url's :sha? Or is that just speculative "t.d.a. could maybe do this perhaps"? #2019-12-02 23:28:03 seancorfield (I'm guessing that it's speculative and in many cases it could not resolve that anyway because there isn't always valid tag/scm information in the POM anyway) #2019-12-02 23:47:38 alexmiller There usually is scm info #2019-12-02 23:47:55 alexmiller I have not yet implemented this but all the pieces are already there #2019-12-03 00:01:07 didibus I see, so local deps always wins, and in the case of conflicting local deps it would error. #2019-12-03 00:01:28 didibus But maven and git can both use the commit sha to find out which one is newest #2019-12-03 00:10:42 seancorfield @didibus it would be in pom.xml not the manifest #2019-12-03 00:11:39 seancorfield for example
<scm>
<url></url>
<connection>scm:git:</connection>
<developerConnection>scm:git:@github.com/seancorfield/next-jdbc.git</developerConnection>
<tag>05a6d8a47fcea7dc2928e5e2c176310fb7205e0c</tag>
</scm>
(from next.jdbc's pom.xml that is used for deployment to Clojars) #2019-12-03 00:12:55 didibus I thought it was common to put tag to HEAD ? #2019-12-03 00:13:25 didibus Also, what does maven do with that? Is the pom.xml uploded to Clojars as well? #2019-12-03 00:15:36 seancorfield Yes, the pom.xml is what tells Clojars how to link to the commit that is tied to the release. #2019-12-03 00:16:27 seancorfield It can either be a specific tag or a SHA. #2019-12-03 00:16:29 didibus I see #2019-12-03 00:16:52 seancorfield (I happen to use SHAs, just because I have a deploy script that does that automatically) #2019-12-03 00:17:05 didibus Sounds like days worth of fun debugging conflicts to be honest 😛 But hopefully it mostly works itself out in practice #2019-12-03 00:17:27 seancorfield Why would you think that? #2019-12-03 00:17:53 seancorfield Clojars shows which commit is referenced in pom.xml #2019-12-03 00:19:22 didibus Well, maybe I'm wrong. But I can see the tag not mapping to the correct version, and issues incurring. Though I guess there's nothing to do about that #2019-12-03 00:19:58 seancorfield Sure, you can "lie" in pom.xml if you don't have some automated release process... #2019-12-03 00:20:18 didibus And what does happen if you set HEAD as the tag? #2019-12-03 00:20:34 didibus Is that automatically replaced by the HEAD sha at time of deploy? #2019-12-03 00:20:54 seancorfield Deployment is independent of Git (so, no). #2019-12-03 00:21:13 seancorfield (at least, that is my understanding) #2019-12-03 00:22:03 seancorfield Certainly, you can mess up the <scm> part of pom.xml -- when I first started publishing to Clojars via clojure-generated POM files, I messed up several times. Hence, automation 🙂 #2019-12-03 00:22:45 seancorfield cljdoc also relies on pom.xml being accurate so it can pull the correct version of your repo from GitHub to generate docs that match the release artifact... #2019-12-03 00:23:59 didibus It says: tag: Specifies the tag that this project lives under. HEAD (meaning, the SCM root) should be the default. #2019-12-03 00:27:45 didibus Anyways, it does seem more standard then I thought. It should be a pretty nice way to handle cross resolver conflicts. #2019-12-03 00:28:13 didibus I guess a local deps could also use the pom.xml found in the directory for conflict resolution ? #2019-12-03 00:31:11 alexmiller it could if there is one #2019-12-03 00:31:28 alexmiller although I think the chances that value is meaningful+correct in a local dep is low #2019-12-03 00:32:07 alexmiller vs the rev in an artifact, which is almost always updated during deployment #2019-12-03 00:32:50 alexmiller open question what to do for a local dep that also happens to be a git repo #2019-12-03 00:33:11 alexmiller I would need to think about that for a while and make some tables #2019-12-03 00:35:11 didibus hammock #2019-12-03 00:35:18 alexmiller there are requests for git deps on the file system too (which we don't support right now) so that intersects #2019-12-03 00:37:23 didibus The world of dependency management is hard. I've lost so many hours of my life managing dependencies 😛 At least Clojure libs rarely break anything, so which version ends up chosen doesn't matter as much 😛 #2019-12-03 00:39:09 alexmiller yeah, I've been doing this stuff for 22 years in the Java universe. those early years pre-Maven were kinda weird. #2019-12-03 00:40:05 didibus I do love how tools.deps resolves conflicts by picking latest though. That's a huge improvement for me with regards to Maven. #2019-12-03 00:40:07 seancorfield Somehow I managed to completely skip Maven until I got to Clojure... 👀 #2019-12-03 00:42:26 alexmiller managing your deps manually is a very instructive process (not very efficient in the long run, but doing it a little really gives you an appreciation) #2019-12-03 00:42:41 alexmiller I used (and wrote) a lot of bespoke tools pre-Maven #2019-12-03 00:44:33 didibus I never lived in the pre-maven world, for the better I guess #2019-12-03 10:16:59 niclasnilsson adding libs on the fly in the repl? If I’m not mistaking the commit pointed to in http://insideclojure.org/2018/05/04/add-lib/ is gone, and github says there is no add-lib function or a repl ns in the repo. Any ideas on how people do this nowadays? #2019-12-03 10:18:36 dharrigan and in particular line 139 #2019-12-03 10:18:41 dharrigan you'll see how it is used 🙂 #2019-12-03 10:36:08 niclasnilsson @dharrigan, thanks (I wonder why github didn’t find add-lib in that very repo though?) . But, I don’t get it work though. Installing the deps worked, but I get a NullPointerException when trying to add a lib. Anything you recognize?
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'honeysql {:mvn/version "0.9.8"})
Execution error (NullPointerException) at clojure.tools.deps.alpha.util.maven/make-session (maven.clj:173).
null #2019-12-03 10:39:28 alexmiller add-lib exists only on a branch, has never been integrated on the main line #2019-12-03 10:40:48 niclasnilsson @alexmiller, yep, understood. I used the sha in Sean’s deps.edn right now though #2019-12-03 10:44:24 alexmiller That sha is on the add-lib branch which typically lags master #2019-12-03 10:41:20 niclasnilsson (when I get the NPE) #2019-12-03 10:41:22 alexmiller You should never see that error but I have seen people report this. I’ve never been able to repro #2019-12-03 10:42:03 niclasnilsson Interesting. Any special investigation I can undertake on my env? #2019-12-03 10:43:15 alexmiller It should be impossible by my understanding of the code so clearly some assumption I have is wrong #2019-12-03 10:43:47 niclasnilsson Even more interesting 🙂 #2019-12-03 10:44:02 dharrigan @niclasnilsson seems to work for me! #2019-12-03 10:44:13 dharrigan user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'honeysql {:mvn/version "0.9.8"})
true #2019-12-03 10:44:42 dharrigan Try deleting your .cpcache? #2019-12-03 10:45:01 niclasnilsson Good idea. Will try. (But I’ll save it some place else) #2019-12-03 10:45:04 alexmiller Should t be using cpcache #2019-12-03 10:45:09 dharrigan k #2019-12-03 10:45:56 alexmiller That’s for caching classpaths but you’re not making those here, you’re dynamically updating runtime #2019-12-03 10:46:28 alexmiller What repl env are you in? #2019-12-03 10:46:29 dharrigan 👍 #2019-12-03 10:46:56 alexmiller Not all repls have the right classloader setup for this to work #2019-12-03 10:49:48 niclasnilsson Clearing .cpcaches didn’t work #2019-12-03 10:50:20 alexmiller I’d be shocked if it did :) #2019-12-03 10:50:33 niclasnilsson repl: Vanilla clj #2019-12-03 10:51:00 niclasnilsson (no socket, no nrepl) #2019-12-03 10:51:18 alexmiller Well, don’t know then #2019-12-03 10:52:18 niclasnilsson The docs say “Note that for successful use, you must be in a REPL environment where a valid parent DynamicClassLoader can be found in which to add the new lib urls.“, but I suppose that’s the case when I just fire up clj? #2019-12-03 10:52:29 alexmiller Yes #2019-12-03 10:52:31 niclasnilsson Ok #2019-12-03 10:54:43 niclasnilsson Hmm… It works if I have a deps.edn only contaning
{:deps
{org.clojure/tools.deps.alpha {:git/url ""
:sha "148cab8f154711f8495bd632ebdc53a14dff7f09"}}}
#2019-12-03 10:55:41 niclasnilsson But not in my project, so likely something in there messes it up. I’ll reduce my deps.edn and see if I can find it. #2019-12-03 10:57:11 dharrigan you can do clojure -Stree to get a list #2019-12-03 10:57:24 alexmiller If you have something that doesn’t work now, I’d take that #2019-12-03 10:58:52 dominicm I had a repro for this involving dependency conflicts #2019-12-03 11:00:16 dominicm The session lookup fails because its the wrong class #2019-12-03 11:00:31 dominicm It's one from a different dependency #2019-12-03 11:01:53 dominicm It's not specific to the add lib branch, it's just the only time users interact with maven outside of the clj internal classpath #2019-12-03 11:11:09 niclasnilsson {:deps
{metasoarous/oz {:mvn/version "1.6.0-alpha5"}
org.clojure/tools.deps.alpha {:git/url ""
:sha "148cab8f154711f8495bd632ebdc53a14dff7f09"}}}
and
(require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
(add-lib 'honeysql {:mvn/version "0.9.8"})
causes the NullPointerException.
Unfortunately oz pulls in half the world… #2019-12-03 11:13:45 niclasnilsson Ah, I see @dominicm already pointed out oz and the cause. #2019-12-03 11:15:01 dominicm @alexmiller let me know if there's anything I can do to help with explaining or documenting this. I spent a lot of effort debugging it, and I hate for that to be a waste. #2019-12-03 11:15:26 alexmiller Oh interesting, I had not seen all that debugging before #2019-12-03 11:15:43 alexmiller Do you have jira access? #2019-12-03 11:15:47 dominicm You helped me with it 😜 #2019-12-03 11:15:58 dominicm I do, yes. #2019-12-03 11:16:30 alexmiller If you could file a tdeps ticket, that would be great for tracking #2019-12-03 11:16:55 alexmiller I don’t think there is one #2019-12-03 11:20:44 alexmiller Certainly oz should not be publishing an uberjar, that’s evil #2019-12-03 11:21:11 alexmiller I’d like to be able to fail better in tdeps though #2019-12-03 11:26:13 dominicm oz shouldn't be doing an uberjar, but this is unrelated to that. #2019-12-03 11:27:12 dominicm the problem is that maven-aether-resolver and maven-resolver-provider conflict. I think oz gets the former through pomegranate, but icr right now, will double check before I file the jira. #2019-12-03 11:27:44 dominicm @alexmiller I think I did mention the resolution in this channel, is there a better place to mention that kind of thing to make sure you don't miss it next time? #2019-12-03 11:34:54 alexmiller Jira #2019-12-03 11:29:55 dominicm As a total aside, on my wishlist if for maven to support "provides", as in "juxt/tools.namespace" provides "org.clojure/tools.namespace", so there should only be one of them in the classpath (same for things where a -complete artifact is provided such as antlr. There's N artifacts and a "bundled" artifact too.) #2019-12-03 21:55:27 markbastian I'm having a deps issue with (drumroll, please), jackson. I have com.fasterxml.jackson.core/jackson-core {:mvn/version "2.10.1"} in my :deps map, but when I do an ion-dev push it reports a conflict of com.fasterxml.jackson.core/jackson-core #:mvn{:version "2.9.8"}. This manifests itself when my ns gets loaded with the error "com.fasterxml.jackson.core.exc.InputCoercionException," which is a 2.10 addition. I tried adding the library to :override-deps instead, without success. Any tips on what I am doing wrong (besides needing jackson)? #2019-12-03 22:00:06 alexmiller can you share your ion-dev alias? #2019-12-03 22:00:20 alexmiller specifically, is it using :extra-deps or :deps? #2019-12-03 22:00:55 alexmiller and if it's using :extra-deps, can you switch it to :deps? (need newish version of clj for that to work) #2019-12-03 22:01:56 markbastian Sure, latest attempt:
{:aliases
{:ion-dev
{:deps {com.datomic/ion-dev {:mvn/version "0.9.247"}}
:main-opts ["-m" "datomic.ion.dev"]}}
:paths ["src" "resources"]
:deps {com.cognitect/anomalies {:mvn/version "0.1.12"}
com.datomic/client-cloud {:mvn/version "0.8.78"}
com.datomic/ion {:mvn/version "0.9.35"}
org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/data.json {:mvn/version "0.2.6"}
metosin/ring-http-response {:mvn/version "0.9.1"}
ring/ring-defaults {:mvn/version "0.3.2"}
ring/ring-json {:mvn/version "0.5.0"}
metosin/jsonista {:mvn/version "0.2.5"}
metosin/reitit {:mvn/version "0.3.10"}
javax.xml.bind/jaxb-api {:mvn/version "2.4.0-b180830.0359"}
}
:override-deps {com.fasterxml.jackson.core/jackson-core {:mvn/version "2.10.1"}
com.fasterxml.jackson.core/jackson-databind {:mvn/version "2.10.1"}}
:mvn/repos {"datomic-cloud" {:url ""}}} #2019-12-03 22:02:29 alexmiller so the error is not during push, it's when your code is run in the ion #2019-12-03 22:03:12 alexmiller right? #2019-12-03 22:03:39 alexmiller that :override-deps in your deps.edn isn't doing anything - that only works in an alias #2019-12-03 22:04:09 alexmiller but you can just add those to your main :deps #2019-12-03 22:04:15 markbastian Right, I get a warning when I push stating that I have the dependency conflict on 2.9.8 and the error is at runtime. I am assuming this is because the ion is using the older version without the InputCoercionException class. #2019-12-03 22:04:45 markbastian Yeah, that override-deps was just my attempt to get the override to work. It was in deps before. #2019-12-03 22:04:57 alexmiller {:aliases
{:ion-dev
{:deps {com.datomic/ion-dev {:mvn/version "0.9.247"}}
:main-opts ["-m" "datomic.ion.dev"]}}
:paths ["src" "resources"]
:deps {com.cognitect/anomalies {:mvn/version "0.1.12"}
com.datomic/client-cloud {:mvn/version "0.8.78"}
com.datomic/ion {:mvn/version "0.9.35"}
org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/data.json {:mvn/version "0.2.6"}
metosin/ring-http-response {:mvn/version "0.9.1"}
ring/ring-defaults {:mvn/version "0.3.2"}
ring/ring-json {:mvn/version "0.5.0"}
metosin/jsonista {:mvn/version "0.2.5"}
metosin/reitit {:mvn/version "0.3.10"}
javax.xml.bind/jaxb-api {:mvn/version "2.4.0-b180830.0359"}
com.fasterxml.jackson.core/jackson-core {:mvn/version "2.10.1"}
com.fasterxml.jackson.core/jackson-databind {:mvn/version "2.10.1"}}
:mvn/repos {"datomic-cloud" {:url ""}}}
#2019-12-03 22:06:18 alexmiller ok, well then I would send you to #datomic - this is not the first time they've seen something like this, but I don't know the recommended path forward. I think you're right on with what the problem is. #2019-12-03 22:06:23 markbastian I'll give that shot and report the output. I think I had done that before. BTW, the jaxb addition was because of this warning when I do the ion push: [s3-transfer-manager-worker-6] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant #2019-12-03 22:06:53 markbastian Cool, let me give this a shot and I'll let you know what happened then bounce over to datomic. #2019-12-03 22:06:58 alexmiller yeah, that's likely due to being on java 9+ #2019-12-03 22:07:05 alexmiller it's included in java 8 #2019-12-03 22:07:24 alexmiller but they've been shedding all those j2ee libs from the jdk #2019-12-03 22:07:25 markbastian Is the datomic runtime 8 or 9? #2019-12-03 22:07:32 alexmiller dunno #2019-12-03 22:07:42 alexmiller I mean, it must be 8 or 11 #2019-12-03 22:07:55 alexmiller 9 and 10 are dead and I can't imagine they're using those, so I'd guess 8 #2019-12-03 22:09:43 markbastian Full output:
Downloading: com/datomic/java-io/0.1.11/java-io-0.1.11.pom from
(cognitect.s3-libs.s3/upload "datomic-code-d70d7189-fa46-4cd5-8004-423b714e486b" [{:local-zip ".datomic-ions/datomic/apps/ion-repl/stable/ff0ed784b940e81fa02f8a0808a3fc06719bfc1b.zip", :s3-zip "datomic/apps/ion-repl/stable/ff0ed784b940e81fa02f8a0808a3fc06719bfc1b.zip"}] {:op :push})
[s3-transfer-manager-worker-6] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant
{:rev "ff0ed784b940e81fa02f8a0808a3fc06719bfc1b",
:deploy-groups (ion-repl-Compute-1TPY2S14U7NT9),
:dependency-conflicts
{:deps
{com.cognitect/transit-java #:mvn{:version "0.8.311"},
org.clojure/clojure #:mvn{:version "1.10.0"},
commons-codec/commons-codec #:mvn{:version "1.12"},
com.datomic/client-api #:mvn{:version "0.8.37"},
com.fasterxml.jackson.core/jackson-core #:mvn{:version "2.9.8"},
com.datomic/client #:mvn{:version "0.8.86"},
org.msgpack/msgpack #:mvn{:version "0.6.10"},
com.datomic/client-impl-shared #:mvn{:version "0.8.69"},
com.cognitect/transit-clj #:mvn{:version "0.8.285"},
com.cognitect/s3-creds #:mvn{:version "0.1.23"},
com.datomic/client-cloud #:mvn{:version "0.8.80"},
com.amazonaws/aws-java-sdk-kms #:mvn{:version "1.11.479"},
com.amazonaws/aws-java-sdk-s3 #:mvn{:version "1.11.479"}},
:doc
"The :push operation overrode these dependencies to match versions already running in Datomic Cloud. To test locally, add these explicit deps to your deps.edn."},
:deploy-command
"clojure -A:ion-dev '{:op :deploy, :group ion-repl-Compute-1TPY2S14U7NT9, :rev \"ff0ed784b940e81fa02f8a0808a3fc06719bfc1b\"}'",
:doc
"To deploy to ion-repl-Compute-1TPY2S14U7NT9, issue the :deploy-command"} #2019-12-03 22:10:14 markbastian I'll try the deploy, but I think it's going to be broken. I'll try #datomic. Thanks! #2019-12-03 22:10:34 alexmiller yeah, I mean it's telling you that in the result there #2019-12-03 22:10:51 markbastian Right. "We're using this library that hasn't got a class you're going to need." #2019-12-03 22:52:48 Alexis Vincent @alexmiller Hi there Alex, thanks for building/maintaining deps. And all the rest you do for the clj community. Much appreciation!
Had a couple of questions.
1. I’m trying to use tools.deps as a lib. But have an issue using resolve-deps . One of the deps.edn files uses a package from a mvn repo, where the repo is declared in the deps.edn file, and where the cli picks it ub correctly. But it blows up for me when using the lib with:
Could not find artifact io.confluent:kafka-schema-registry-client:jar:5.3.0
in central ()
#2019-12-03 22:53:54 Alexis Vincent im calling it using
(-> {:deps {'root/root {:local/root "<path to root project>"}}
:mvn/repos mvn/standard-repos}
(deps/resolve-deps nil
{:trace true}))) #2019-12-03 22:55:18 Alexis Vincent the required file includes the mvn repo as
:mvn/repos
{"confluent" {:url ""}} #2019-12-03 22:58:05 Alexis Vincent 2. I have a case where that file adds a jar to its paths. But when using juxt’s pack, it thinks the jar lives at one of the transative local/root dependencies dirs. I dont know if this is a pack problem or something with tools.deps. But im guessing deps. #2019-12-04 08:01:48 dominicm I don't fully follow this, sorry. Can you expand? #2019-12-03 23:00:14 Alexis Vincent How do I best give you info to help fix the bug? I’ve managed to work around it for now using depstar #2019-12-04 03:38:34 alexmiller @mail024 so the mvn/repos declaration is in a transitive deps.edn? I suspect that's probably not handled right now #2019-12-04 03:56:51 alexmiller I added a note there #2019-12-04 14:54:00 FiVo T #2019-12-04 14:55:44 FiVo A sorry wrong channel. #2019-12-04 15:15:15 alexmiller works for me? #2019-12-04 15:16:01 alexmiller I guess he fixed it #2019-12-04 15:16:16 alexmiller I must have accidentally nuked it yesterday #2019-12-04 15:16:21 FiVo Yes it's up again. #2019-12-06 20:04:08 dominicm I just responded to this pack issue which I think is a tdeps bug, but I don't know much about maven/pom.xml, so it might be helpful to @drewverlee if someone who was more familiar took a look: https://github.com/juxt/pack.alpha/issues/68#issuecomment-562717952
Essentially it doesn't seem like the sourceDirectories in pom.xml are being read. #2019-12-06 20:13:27 drewverlee On my phone atm, I'll be back home on Sunday and I'll look into this. #2019-12-06 20:13:36 drewverlee Thanks for looking into the issue. #2019-12-06 20:15:58 dominicm Looks like I'm wrong, I'll take another look in a minute :) #2019-12-06 20:09:06 alexmiller I don't understand what the expected and actual behavior is? #2019-12-06 20:09:30 alexmiller I tried
clj -Spath -Sdeps '{:deps {integrant {:git/url "" :sha "c673b85130e553feec6d4c5d2d1ec773a49c929c"}}}'
#2019-12-06 20:09:41 alexmiller and got
src:/Users/alex/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/alex/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src:/Users/alex/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/src/main/clojure:/Users/alex/.gitlibs/libs/integrant/integrant/c673b85130e553feec6d4c5d2d1ec773a49c929c/resources:/Users/alex/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/alex/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/alex/.m2/repository/weavejester/dependency/0.2.1/dependency-0.2.1.jar
#2019-12-06 20:10:14 alexmiller what is expected that's different? #2019-12-06 20:11:12 alexmiller that includes the srcDirectory (src) #2019-12-06 20:12:05 dominicm Oh. So it does. My mistake. Reading long strings is hard. I missed that. #2019-12-06 20:25:00 ghadi I feed them to
clj -Spath | tr ':' '\n' #2019-12-06 20:30:16 sogaiu nice #2019-12-06 20:49:33 alexmiller 👍 #2019-12-06 20:52:23 dominicm I need to upgrade to the 0.8.x series really, I'm just not keeping on top of this stuff very well at the moment. Life is busy. #2019-12-06 20:56:25 alexmiller well, should be no breaking changes :) #2019-12-06 22:25:20 seancorfield @alexmiller I noticed that recent versions of clojure/t.d.a. actually strip existing newlines from POM files -- was that intentional when you fixed TDEPS-29? #2019-12-06 22:26:13 seancorfield In my pom.xml files, I tend to have blank lines separating sections at the top-level. clojure -Spom now removes those blank lines. Which is really annoying -- it should not mess with the existing layout. #2019-12-06 22:39:49 alexmiller It was not intentional but makes sense why that would be #2019-12-06 22:40:34 alexmiller As you know, parsing, modifying, then emitting xml is tough to do w/o something like this happening #2019-12-06 22:41:16 alexmiller (Really, xslt is a better tool for this job but I didn’t have much luck going down that route) #2019-12-06 22:41:48 alexmiller Patches welcome :) #2019-12-06 23:02:46 dominicm Is xslt whitespace preserving? #2019-12-06 23:41:45 seancorfield I don't consider it quite annoying enough to create a JIRA issue so I'll just learn to live with it 🙂 #2019-12-06 23:44:41 alexmiller xslt has a lot of options, not sure. I did look seriously at it back when I wrote the pom sync, but that was a couple yrs ago, don't remember the tradeoffs now. #2019-12-07 06:45:53 pyr Hi, I know it's not tools.deps per-se #2019-12-07 06:46:24 pyr but related, I discovered pulling private repositories with gitlibs was not possible with HTTP auth #2019-12-07 06:46:41 pyr when using github tokens for automation tooling this can be an issue for finer grained repo access control #2019-12-07 09:16:08 alexmiller We don't want any credentials to show up in deps.edn - with this you'd have plaintext passwords in git urls wouldn't you? #2019-12-07 09:25:56 alexmiller If we're using git auth, you have a few different choices - I don't think anyone wants to manually type in passwords every time (or any time) and that ruins pretty much any automation workflow, but you can use "store" mode (stored in cleartext file), or "oskeychain" mode on mac, or the "Git Credential Manager for Windows" mode on windows which uses Windows Credential Store, or I think there may be other more generic password oracle options for Git. #2019-12-07 09:27:07 alexmiller If we're using jgit, then it's a matter of where we can read pws from - again you talk to os-dependent things like the keychain, credentials store, etc but we're probably building that ourselves (or maybe something exists, don't know) #2019-12-07 09:27:35 alexmiller But my main constraint is, don't put it in deps.edn. #2019-12-07 11:36:01 pyr @alexmiller OK understood. A possible alternative is to go with the NetRCCredentialsProvider. Shelling out to git would solve these issues #2019-12-07 11:36:54 pyr I'm using tools.gitlibs here from outside of tdeps which is why having the tokens in the config makes sense to me in this case but I get that for tdeps not necessarily. #2019-12-07 11:39:10 pyr I'll drop my commit for now, thanks for the clarification #2019-12-07 11:40:07 pyr Looking at the issue, it's indeed shell-out to git or a custom CredentialsProvider implementation which looks for credentials.edn #2019-12-11 13:11:40 borkdude it is possible to generate a classpath without clojure and spec itself on it? #2019-12-11 13:12:23 borkdude e.g. here:
$ clojure -Srepro -Sdeps '{:deps {myscript {:git/url "" :sha "89f66ba2101180a98acf88441693840875288ec9"}}}' -Spath
I want only the git dep to be on the classpath #2019-12-11 13:12:42 borkdude but I'm getting:
src:/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/borkdude/.gitlibs/libs/myscript/myscript/89f66ba2101180a98acf88441693840875288ec9:/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
#2019-12-11 13:13:31 borkdude I want to use tools.deps to create a classpath and download deps for a clojure interpreter tool I'm writing #2019-12-11 13:14:40 alexmiller Via clj, no (due to the install deps.edn). Via tools.deps, yes. #2019-12-11 13:15:04 borkdude so I'd have to write my own tool that uses tools.deps as a lib? ah #2019-12-11 13:15:10 alexmiller Yes #2019-12-11 13:15:31 alexmiller clj is a Clojure runner #2019-12-11 13:15:48 alexmiller It assumes you’re using Clojure #2019-12-11 13:16:05 borkdude ok #2019-12-11 13:29:34 borkdude btw I could see this feature be useful for other environments like self-hosted Clojure / planck, which is already using the clojure like this #2019-12-11 13:29:50 borkdude I'll ask in their channel how they solve this "problem" (more of an optimization really) #2019-12-11 13:46:51 alexmiller You can run tools.deps from clj btw - there is a built in alias for it -A:deps #2019-12-11 13:48:00 alexmiller It’s not exactly a 1 liner but the tools are there (really depends precisely on what you want to do) #2019-12-11 14:00:38 borkdude I'll figure something out. Btw, dirty hack: {:deps {org.clojure/clojure {:local/root "."}}} #2019-12-11 14:12:20 alexmiller really ugly 1-liner: #2019-12-11 14:12:24 alexmiller clj -A:deps -e "(println ((requiring-resolve 'clojure.tools.deps.alpha/make-classpath) ((requiring-resolve 'clojure.tools.deps.alpha/resolve-deps) {:deps {'org.ow2.asm/asm {:mvn/version \"7.2\"}} :mvn/repos @(requiring-resolve 'clojure.tools.deps.alpha.util.maven/standard-repos)} nil) nil nil))" #2019-12-11 14:12:44 alexmiller prints to stdout /Users/alex/.m2/repository/org/ow2/asm/asm/7.2/asm-7.2.jar #2019-12-11 14:13:03 alexmiller asm just used as an example #2019-12-11 14:15:09 alexmiller you could also slurp from deps.edn or whatever (some tools for that stuff in clojure.tools.deps.alpha.reader) #2019-12-11 15:10:22 borkdude nice, thanks! #2019-12-11 15:36:18 borkdude I think I'll just use the existing tools though, the entire tooling is already there, caching and all, it'd be a lot of work and error-prone to replicate this. I'll just skip over some known deps that aren't interesting for the interpreter #2019-12-11 15:37:15 borkdude The idea is that the interpreter is a drop-in replacement for a program you can otherwise run with normal Clojure (give or take some reader conditionals) #2019-12-11 15:37:41 borkdude so re-using most of what's there makes also sense from that perspective #2019-12-11 15:38:30 alexmiller gotcha - to some degree the :classpath-overrides alias option is a good match for this #2019-12-11 15:38:48 alexmiller you could just override the classpath part for org.clojure/clojure to nothing #2019-12-11 15:39:28 alexmiller I guess you'd need to also do clojure's deps though #2019-12-11 15:41:32 alexmiller clj -Sdeps '{:aliases {:bye {:classpath-overrides {org.clojure/clojure "" org.clojure/spec.alpha "" org.clojure/core.specs.alpha ""}}}}' -A:bye -Spath #2019-12-11 15:42:59 borkdude {:aliases {:babashka {:classpath-overrides {org.clojure/clojure nil
org.clojure/spec.alpha nil
org.clojure/core.specs.alpha nil}}}}
$ clj -C:babashka -Spath
src:::
#2019-12-11 15:43:02 borkdude Perfect! #2019-12-11 15:50:08 alexmiller I should probably be killing the nils in there :) #2019-12-11 15:50:15 alexmiller but yeaeh #2019-12-11 15:54:09 alexmiller committed that fix for whenever I do next release #2019-12-11 15:56:15 borkdude maybe remove empty? #2019-12-11 15:56:54 alexmiller ? #2019-12-11 15:57:23 borkdude instead of remove nil? so empty strings are also "killed" #2019-12-11 15:57:40 alexmiller well, would use str/blank? for that case #2019-12-11 15:58:52 alexmiller imo it's bad form to seq-ize strings with empty? vs using a string-specific predicate #2019-12-11 15:59:08 borkdude ok #2019-12-11 15:59:57 alexmiller done #2019-12-11 16:02:43 borkdude awesome 🙂 #2019-12-11 16:12:46 borkdude Not that I need it right now, but it is possible to override the tools.deps version for the clojure tool to a git dep on master? #2019-12-11 16:50:48 alexmiller no, not without hacking the script #2019-12-11 16:51:10 alexmiller hacking works fine though, I test stuff that way all the time :) #2019-12-11 16:52:18 alexmiller I guess maybe for this use of -A:deps, you might be able to use :override-deps or :classpath-overrides to make that work #2019-12-11 16:58:40 alexmiller that is, there is no hook to affect the classpath of the jvm run that uses tools.deps.alpha to make the classpath, but you could use the existing alias modifiers to change the classpath of the program you're running if it uses tools.deps.alpha #2019-12-12 17:00:06 scottlowe Hi there. I’m working on a project that has a dependency on a large repo of test data, which we don’t want comitted in the main project repo.
Once retrieved, the directory is 295 MB uncompressed on disk. We are seeing OOMEs when tools.deps is retrieving via tools.gitlibs: #2019-12-12 17:00:16 scottlowe Cloning:
#2019-12-12 17:00:30 scottlowe This error never occurs locally on macos dev machines, but when running on a Travis CI Linux VM. Ubuntu Bionic.
I note that tools.gitlibs “0.2.64” is using jgit “4.10.0.201712302008-r”. #2019-12-12 17:00:46 scottlowe Digging a bit futher it looks like jGit is on v5.x series and tools.gitlibs is on the v4.x series. Notice that some recent releases of jGit have memory leak fixes. Hard to say if this relates directly to our issues, though. #2019-12-12 17:01:00 scottlowe Is this a bug that I should create a ticket for? Or are we using tool.deps outside of its parameters of intended use? #2019-12-12 17:02:51 scottlowe I’ve tried giving the JVM a good deal of memory, and when that didn’t work, I’ve tried restricting memory to a smaller amount (1.2 GB) so that the JVM can better plan its memory use… but neither approach has worked. #2019-12-12 17:03:37 ghadi @scottlowe how specifically did you give the JVM more memory? #2019-12-12 17:04:54 scottlowe Hi Ghadi. I used an env var e.g.: JAVA_OPTS=-Xmx1280m. Less than was available on the Linux instance. I wanted to avoid the Linux OOM-Killer… which may or may not have been related. #2019-12-12 17:08:05 ghadi clearing up a couple misconceptions:
1) clj launches two JVMs, one to clone the repos, then another for your project. To modify the JVM for the first one, you need to change this line https://github.com/clojure/brew-install/blob/1.10.1/src/main/resources/clojure#L314 in /usr/local/bin/clojure or wherever your script lives
2) JAVA_OPTS isn't a thing, but a convention in some scripts -- not honored by clj #2019-12-12 17:09:15 scottlowe Wow. I didn’t know that. Thanks for the education 🙂 #2019-12-12 17:09:21 ghadi np #2019-12-12 17:10:02 ghadi try bumping up from 256m and your problem may (?) go away -- definitely worth a issue though, right @alexmiller? #2019-12-12 17:10:31 scottlowe I will go and try some different memory allocations. Could you just confirm that tools.gitlibs should be able to handle this size of dependency, if conditions are right? #2019-12-12 17:11:18 ghadi not sure what you want me to confirm 🙂 your analysis of the issue seemed right #2019-12-12 17:12:09 ghadi bumping JGit is on the radar, as well as maybe making that whole subsystem more resilient #2019-12-12 17:13:01 dominicm Seems like loading the code into memory would be a mistake somewhere in the pipeline. #2019-12-12 17:13:12 dominicm It can be written straight to disk. #2019-12-12 17:13:51 ghadi you're making an assumption that it's loading into memory #2019-12-12 17:13:54 scottlowe Yeah. I was wondering about that… should be able to stream any amount of data straight through. Thanks. I’ll go away and experiment, then. Will report back either way… but it may take a few days to be sure that the memory tweaks have worked because we don’t see it every time, because of CI dependency caching etc. #2019-12-12 17:14:05 ghadi it might be a large git pack file being decompressed #2019-12-12 17:38:56 alexmiller I have some tabs and editors open working on validating the latest versions of jgit, but it's a notch or two down the stack right now. Kind of an interim step towards comparing latest jgit vs shelling out to git for https/ssh access #2019-12-12 17:39:41 alexmiller generally, I'd expect the jvm heap on the cp-making jvm to grow quite a bit larger, but it depends what jvm and version you're using #2019-12-12 18:21:12 scottlowe Ahh. JVM versions... that’s another longer story. In our dev and CI environment we’re currently restricted to using OpenJDK 8 because of a DB dependency. Our Clojure code should be fine running under version 11 and we hope to move to OpenJDK 11 early next year. It’s possible we could use multiple JVM versions in the meantime (to start the DB under a seperate JVM), using a version switcher or something, but that might be asking for trouble as well as more complexity! #2019-12-12 18:21:13 scottlowe Thanks for the comments Alex. I had heard about the shelling out approach for future versions of tools.gitlibs, and I’m hopeful about that approach. I look forward to your findings. :crossed_fingers: #2019-12-13 04:48:25 mattyulrich Hi all - I’m sure this has been asked before but I just can’t find the answer. Is there a way to signal in my deps.edn that one of my deps is a bom and not a jar? #2019-12-13 05:10:15 mattyulrich Or, actually more specifically, I seem to have problem with a bom as transitive deps. #2019-12-13 08:13:52 dominicm What's a bom? #2019-12-13 08:56:53 dominicm Oops. I was totally expecting it to be a typo of pom. #2019-12-13 09:06:45 andy.fingerhut You may be right, but bom does have a relevant meaning here. #2019-12-13 09:17:43 plexus not to be confused with a byte order mark #2019-12-13 14:43:46 mattyulrich Hey all - actually noticed some updated support for this. Switched to the latest tools.deps and the transitive dep was resolved.. yeah - I was talking about a “bill of materials” dep.. #2019-12-13 11:02:58 codeasone What route should I take to get the equivalent of lein uberjar with :aot :all using the Clojure CLI tools? #2019-12-13 11:04:21 codeasone I'm presently using depstar but there doesn't appear to be an equivalent option #2019-12-13 17:00:27 seancorfield We use depstar at work to build uberjars for production deployment and we do not AOT anything. #2019-12-14 00:23:39 codeasone Thanks for replying @U04V70XH6. Along with deploying our uberjar to ECS within a docker image, we also invoke it multiple times within our CI pipeline. My principle motivation was to shave some time off the CI runs, and additionally plain curiosity around why the facility wasn't being supported yet. #2019-12-14 00:25:55 seancorfield Fair enough. If you (manually) create a classes folder in your project and add it as an :extra-paths entry when building your uberjar, you can run clojure -e "(compile,'my.entry.point)" as a way to compile the entry namespace (`my.entry.point` here) and its transitive deps. #2019-12-14 00:27:02 codeasone Great tip: thanks! #2019-12-14 00:32:57 seancorfield depstar tries hard to avoid messing with your project or your classpath: it just packages up whatever you tell it to, using the classpath as constructed by clojure using whatever options you've provided. Because compile requires that classes exists -- or else you can use binding to tell the compiler to put class files somewhere else, that must still exist -- and that the output path of the compiler needs to be on the classpath for depstar to include those files, it just doesn't seem like something depstar itself should try to do. #2019-12-14 00:37:37 seancorfield Note that the depstar README does address this case https://github.com/seancorfield/depstar in terms of Main-Class in the manifest but it stops short of explaining specifically how to compile code and get the classes onto your classpath. #2019-12-14 18:18:07 codeasone Thanks for clarifying. #2019-12-13 11:54:58 kszabo AOTing is not recommended anymore unless you deeply care about startup perf #2019-12-13 12:00:41 borkdude if you have an :aliases alias in ~/.clojure/deps.edn and one in your project deps.edn, it seems they are not merged? #2019-12-13 12:01:25 borkdude e.g. putting:
:babashka {:classpath-overrides {org.clojure/clojure nil
org.clojure/spec.alpha nil
org.clojure/core.specs.alpha nil}}
in ~/.clojure/deps.edn
and in the project.deps::
:babashka {:classpath-overrides {my_gist_script nil}}
does not seem to merge #2019-12-13 12:50:32 alexmiller They merge-with merge at the aliases level iirc #2019-12-13 14:53:46 mattyulrich I’m sure this has been asked before, but can’t seem to find it. When within a running clj repl, is it possible to resolve new deps; kind of like boot-clj set-env! ? #2019-12-13 15:13:02 alexmiller there's a speculative feature called add-lib in a branch for this #2019-12-13 15:13:25 alexmiller (sha in that article is out of date) #2019-12-19 19:35:19 vinnyataide where do I find the correct sha? #2019-12-19 19:50:56 alexmiller it was pretty far out of date so I just merged it in 19d197ab221d37db750423eb970880cb87a91100 #2019-12-19 20:08:53 vinnyataide thanks #2019-12-13 15:21:30 mattyulrich Wow - this is great! Thanks Alex! #2019-12-13 15:24:57 alexmiller I think we will probably add this, but some of the details are likely to change #2019-12-13 17:01:35 seancorfield @codeasone why do you want to AOT? #2019-12-13 17:02:29 alexmiller it is perfectly reasonable to AOT a final uberjar for deployment #2019-12-13 18:48:14 hlolli I asked this question few months ago, but since that time, did anyone write a lein-ancient equivalent for tools.deps? #2019-12-13 18:48:14 hlolli I asked this question few months ago, but since that time, did anyone write a lein-ancient equivalent for tools.deps? #2019-12-13 18:58:24 seancorfield We've been using Depot very happily (although I've recently developed a bash script that does crazy stuff with clojure -Stree and a synthesized deps.edn file to get outdated deps in a way that suits us better) #2019-12-13 19:16:49 hlolli wow nice, essential tool in any toolkit. I knew someone had made it by now 🙂 #2019-12-13 19:16:55 hlolli thanks #2019-12-13 19:19:13 borkdude ❤️ depot #2019-12-13 19:20:10 borkdude @U04V70XH6 care to share the bash script? I'd be curious if I can port it to babashka and make a nice example out of it 🙂 #2019-12-13 19:22:27 seancorfield Trust me when I say it's "crazy stuff" -- and it relies very heavily on our custom build shell script that drives our use of clojure... #2019-12-13 19:22:38 borkdude ok #2019-12-13 19:25:34 seancorfield The TL;DR is that it runs clojure -Stree to a file, runs that through sed etc to produce a temporary deps.edn with "RELEASE" for all of the dependency versions (that were :mvn/version explicit versions), then runs clojure -Stree on that, and then it compares the output of both trees, with some fudging to account for only the top two levels of dependencies... #2019-12-13 19:26:21 borkdude sounds like a good hack to me 🙂 #2019-12-13 19:33:01 seancorfield Running Depot reliably on our huge monorepo and getting all the outdated deps that matter and avoiding the noise from transitive deps that I don't care about was very tricky, especially since we use CLJ_CONFIG to setup a baseline deps.edn with :override-deps for a lot of stuff... #2019-12-13 19:36:37 seancorfield ...any t.d.a. tooling that tries to read and process deps.edn files in isolation can run into problems with our setup -- tooling that uses the same process to read/merge the environment as clojure itself does works better for us. #2019-12-13 20:04:35 seancorfield Wow, hahaha!! Important to note that your code won't work with any deps.edn that relies on ~/.clojure/deps.edn (or CLJ_CONFIG) to override/default any deps... 🙂 #2019-12-13 20:05:12 borkdude yeah, this is just a quick sketch which should be tuned to the needs of the scripter #2019-12-13 20:06:39 seancorfield It just goes to show what can easily be done -- and it would work for "most people" I suspect. #2019-12-13 20:06:47 borkdude probably one could come up with a way to use git to inspect newer shas as well #2019-12-13 20:07:26 borkdude but that's an exercise for the reader 🙂 #2019-12-13 20:11:24 borkdude One could even use this for project.clj by just parsing the deps from it, generate a deps.edn and then run this script #2019-12-13 21:16:04 scottlowe @ghadi I’m reporting back on yesterday’s tools.gitlibs OOME issue, which was seen when downloading large repo dependencies: bumping JVM Xms from 256m to 512m seems to have worked. I’ve not seen a failure yet, anyway. Thanks again for your instantaneous help. #2019-12-13 21:16:35 ghadi No problem. What jvm version and vendor do you use btw? #2019-12-13 21:18:57 scottlowe For the CI environment it’s OpenJDK 1.8.0_222 #2019-12-13 21:28:28 alexmiller seems pretty weird that jvm wouldn't grow to 512m #2019-12-13 21:28:51 alexmiller I wonder if you're getting client jvm by default instead of server #2019-12-13 21:31:47 scottlowe Indeed - I had wondered myself why this would be so; I shouldn’t need to babysit the jvm (?); but it does seem to be the server jvm: #2019-12-13 21:31:48 scottlowe $ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
#2019-12-13 21:32:44 alexmiller using same version locally, I get an initial of 256m and max of 4g with no settings #2019-12-13 21:33:04 alexmiller with -Xms256m as set in the script, I see the same #2019-12-13 21:35:21 alexmiller however, the max is dependent on the available ram - I think it's 1/4 total ram by default #2019-12-13 21:35:24 scottlowe I agree that it is odd. This is on a Travis CI instance: Ubuntu 16.04.6 LTS (xenial). We hope to move to more recent versions of the JVM and Ubuntu soon. We’ve never seen this issue on developer machines (macos). The CI environment is more memory constrained than our dev machines. 7.5 GB, I think; but some of that memory is taken up by docker processes, as well as a JVM database process. #2019-12-13 21:38:46 andy.fingerhut Maybe the Travis CI instance has 1G RAM default? #2019-12-13 21:40:56 alexmiller certainly could be something like that #2019-12-13 21:47:15 scottlowe Have checked the Travis CI instance: System Memory: 7680MiB, 64 bits #2019-12-18 23:29:06 caio is there an equivalent of lein's :java-source-paths for deps.edn? #2019-12-18 23:32:49 alexmiller no, clj doesn't support compilation or java source #2019-12-18 23:33:13 alexmiller it's intended for Clojure source only projects that don't require "building" #2019-12-19 08:45:58 vlaaad and yet there are tools already on top of tools-deps that do building and stuff #2019-12-19 08:47:08 vlaaad although they are mostly about making jars from clojure source, haven't seen any that compile java #2019-12-19 09:04:17 stijn is there a limit on the length of paths for tools.deps? when running clj (it's datomic ions push, but I don't think that matters) in Codebuild with local cache enabled for gitlibs it will fail with the following exeption
{:command-failed
"{:op :push :region us-east-1}",
:causes
({:message "Unable to transform path",
:class ExceptionInfo,
:data {:home "/root",
:prefix "/root/.gitlibs/libs",
:resolved-coord
{:git/url "<redacted>",
:sha "e7cc7c48ea0af52f9380c61d2d946cac5d7068e0",
:deps/manifest :deps,
:deps/root "/codebuild/local-cache/custom/b11f68d510f24e13d8132571e19ad44f6bd772898be1fc7b98e76106d7e1ab50/root/.gitlibs/libs/<redacted>/<repo>/e7cc7c48ea0af52f9380c61d2d946cac5d7068e0",
:paths ["/codebuild/local-cache/custom/b11f68d510f24e13d8132571e19ad44f6bd772898be1fc7b98e76106d7e1ab50/root/.gitlibs/libs/<redacted>/<repo>/e7cc7c48ea0af52f9380c61d2d946cac5d7068e0/src"]}}})}
If I run locally I don't see this problem and also when turning off cache for gitlibs, the problem disappears. Codebuild cache will create a symlink of /root/.gitlibs to /codebuild/local-cache/custom/b11f68d510f24e13d8132571e19ad44f6bd772898be1fc7b98e76106d7e1ab50/root/.gitlibs making the directory maybe too long? #2019-12-19 13:39:35 alexmiller I’m not aware of a length issue, should probably ask in #datomic #2019-12-19 13:20:10 vlaaad oh, right! #2019-12-21 18:56:48 practicalli Is it possible to create a project structure as follows with clj-new or any similar tools. Is it just a matter of creating my own template? Or just hacking the project files once its created
advent-of-clojure
|_ src
|_ practicalli
|_ advent_of_code_2019
|_ 01_trinity_of_rocket_equasion.clj
It would have the matching path for test branch too.
/cc @seancorfield #2019-12-21 19:06:42 seancorfield @jr0cket This will get you pretty close to what you want:
clj -A:new app practicalli/advent-of-code-2019.01-trinity-of-rocket-equation
#2019-12-21 19:06:55 seancorfield (! 951)-> tree advent-of-code-2019.01-trinity-of-rocket-equation/
|____
| |____.gitignore
| |____.hgignore
| |____CHANGELOG.md
| |____deps.edn
| |____doc
| | |____intro.md
| |____LICENSE
| |____README.md
| |____resources
| | |____.keep
| |____src
| | |____practicalli
| | | |____advent_of_code_2019
| | | | |____01_trinity_of_rocket_equation.clj
| |____test
| | |____practicalli
| | | |____advent_of_code_2019
| | | | |____01_trinity_of_rocket_equation_test.clj
#2019-12-21 19:17:18 practicalli That great thanks. All I would do is just edit the top level directory name and its perfect.
I was looking at the clj-new documentation and thinking I could also do this with generators. adding new namespaces with clj -A:new -m clj-new.generate ns=practicalli.advent_of_code_2019.01_,,,.clj . This would be handy to script up the adding of a namespace for each day, especially if I wanted a separate -data.clj file for my unique data (keeping the solution namespace cleaner). #2019-12-21 19:30:18 seancorfield Yup, once you have the project folder, you can add new namespaces via the generator logic. #2019-12-25 18:44:39 j Apolgies if this is posted in the wrong group! I created a uberjar using clj -A:new app and clj -A:uberjar myproject.jar alias from @seancorfield’s dot-clojure aliases. I'm getting the following error message though: Error: Invalid or corrupt jarfile myproject.jar
I copied the aliases from the deps.edn from dot-clojure over to the deps.edn in the project to run the :uberjar alias. How do I make a working uberjar? #2019-12-25 19:13:02 seancorfield @watchtheblur The README on depstar explains all that... #2019-12-25 19:14:42 seancorfield At work we run all our uberjars like this: java -cp /path/to/the.jar clojure.main -m entry.point -- which will run entry.point/-main as the "main" function. But the readme explains several other approaches. #2019-12-25 19:16:32 j It worked! Thanks for the assist again! #2019-12-26 14:49:33 rafael Hi. I'm wondering if there are best practices for locally referencing (or referencing via git) a project that relies on gen-class ? If I explicitly compile, everything works out fine, and it was easy to build a task to do that. I'm not sure how to fit it into the development workflow, though. Should I require every contributor to compile locally before working on dependent projects or should I just commit the class files to git and be done with in? Are there alternatives? Can I add a hook somewhere that will compile the gen-class project before starting a repl on a project that depends on it? #2020-12-30 13:49:15 mynomoto We use make to ensure that the necessary classes are compiled when working on dependent projects. #2019-12-28 10:09:28 sveri Hi. I am using cursive which can start an nrepl with deps. Now, when the repl is started I want to switch to a specific namespace and run a function inside.
Using the clojure.main repl I was able to do that by providing a main opts entry point like this:
:aliases {:repl {:main-opts ["-e" "((requiring-resolve 'de.sveri.sg.user/startup))"]}}
Is there something similar for the nrepl? #2020-12-30 00:16:51 seancorfield seancorfield/clj-new {:mvn/version "0.8.1"} -- https://github.com/seancorfield/clj-new/releases/tag/v0.8.1 -- adds pom.xml to app template as well; adds jar/uberjar building aliases to all three built-in templates; expands documentation on the built-in templates how to modify pom.xml generation via the -e / --env options. #2020-12-30 00:19:21 seancorfield In particular, note that projects generated from the app template now let you run clojure -A:uberjar to produce a JAR file that can be run with java -jar -- the :uberjar alias includes a compile call on the main namespace and the classes folder is added to the paths when building the JAR. #2020-12-30 09:49:57 felipebarros I would like to mimic the effect of npm install or lein deps with tools-deps. That is, to fetch the dependencies and exit. Running clj -R:alias always leaves me in a running REPL and I could not find an option in the documentation to do what I want. #2020-12-30 09:54:56 delaguardo clj -Stree will fetch all dependencies, print out complete deps tree and exit #2020-12-30 09:55:44 felipebarros Thanks, that will work :) #2020-12-30 10:24:33 borkdude anything with -Sforce works #2020-12-30 10:25:10 borkdude e.g. -Sforce -Spath also works #2020-12-30 14:32:19 alexmiller These are good answers. you can also clj -e nil #2020-12-30 14:32:36 alexmiller if you want to run something and immediately exit #2020-12-30 16:46:39 felipebarros Thank you :) #2020-12-31 02:06:12 seancorfield @fabrao Boot provides a whole bunch of tools "out of the box" -- although it does not incorporate a test runner (last I checked you needed to add Adzerk's boot-test for that, nor is "boot new" technically built-in but it's an easy command-line dependency to use). As noted, the boot repl allows you to add new dependencies dynamically, and it has some file management tooling built in (such as building jar and uberjar files). #2020-12-31 02:07:23 seancorfield The CLI/`deps.edn` is simple and lightweight, with a range of easy to add tooling for running tests, building jar files, etc. There's an add-lib branch of tools.deps.alpha which you can use -- illustrated in my dot-clojure file -- to add new dependencies dynamically. #2020-12-31 02:08:38 seancorfield I would classify lein and boot as "easy" -- lein more so than boot -- whereas I would classify clj as "simple" (in the general Clojure sense of simple vs easy). #2020-12-31 02:09:11 seancorfield clj is about small, composable tools -- very much in Clojure's "idiomatic world view". #2020-12-31 02:12:02 seancorfield We switched from Leiningen to Boot back in 2015 because we needed more/easier programmability in our build chain. We developed a lot of custom tasks once we switched to Boot -- thousands of lines of dev/test/build code. We kind of lost sight of simplicity there somewhere 🙂 #2020-12-31 02:14:00 seancorfield Our Boot setup already had dependencies in EDN files -- albeit a different format to deps.edn -- so when we were getting ready to switch, I wrote boot-tools-deps (since archived, as a bad idea), as migration step, but in the end we didn't use it. #2020-12-31 02:16:17 seancorfield As part of our switch to deps.edn, we took a long, hard view at all our Boot tasks and decided to streamline/simplify things. We're very happy with the decision to change over. We have just a simple shell script that wraps clojure so we can run multiple CLI commands from a single invocation. #2020-12-31 02:17:16 seancorfield (I will note that we were running into a couple of issues with Boot due to the size of our codebase and our heavy use of "pods", Boot's isolation feature, so we were "pushed" a bit -- it wasn't entirely a "jump") #2020-12-31 02:22:49 seancorfield Happy to chat more via DM any time about tooling, if you want @fabrao (it looks late for you right now -- you're about six hours ahead of me). #2020-12-31 02:31:22 fabrao well, I don´t want to bother you at this time, but any time for me #2020-12-31 02:34:58 fabrao The subject about deps.edn started in Clojure Brazil Telegram Group and your twit shows up and I research about it #2020-12-31 02:41:14 seancorfield It's early evening for me -- feel free 🙂 I'm online about 9am to 11pm Pacific most days 🙂 #2020-12-31 02:44:51 fabrao Hello Sean, how are you? I saw you using REBL instead REPL #2020-12-31 17:23:15 kenny Does anyone recall the solution to "UnknownHostKey: https://github.com. RSA key fingerprint is ...."? We've started getting this on CircleCI. I'm fairly certain I've seen it before and can't remember the solution. The CI has a checkout SSH key. #2020-12-31 17:41:40 kenny Turns out our CI image didn't have ssh or git installed. Interestingly, CircleCI still had some way of getting the repo onto the machine without either of those. Adding them fixed this issue. #2020-12-31 20:23:02 ghadi The CI image doesn’t need git or ssh #2020-12-31 20:23:37 ghadi But it does need the github identity key in the .ssh/known_hosts #2020-12-31 20:23:58 kenny Well in this case it does 🙂 CircleCI adds the checkout ssh key only if ssh is installed. Because ssh wasn't installed, the key was not added. The CLI therefore could not clone the repo. #2020-12-31 20:24:07 ghadi I’d “pin” these in your circleCI yml file #2020-12-31 20:24:10 ghadi k #2020-12-31 19:39:22 seancorfield I finally caved and added an AOT option for uberjars to seancorfield/depstar {:mvn/version "0.4.1"} -- https://github.com/seancorfield/depstar -- there are some caveats: you need to create the classes folder yourself (as a one-off) and you need to ensure that folder is on the classpath when you actually run depstar (the README suggests a :uberjar alias that takes care of that). It only AOTs the -m / --main namespace (but that transitively compiles all namespaces that are directly referenced from it -- it is not "compile all", like Leiningen). #2020-12-31 19:42:45 seancorfield I also updated seancorfield/clj-new {:mvn/version "0.8.2"} -- https://github.com/seancorfield/clj-new -- to take advantage of that for the app template; this also fixes the group/artifact in the pom.xml for generated template projects. #2020-12-31 19:53:21 seancorfield > clojure -A:new app myname/myapp
Generating a project called myapp based on the 'app' template.
> cd myapp/
> clojure -A:uberjar
Compiling myname.myapp ...
Building uber jar: myapp.jar
Processing pom.xml for {myname/myapp {:mvn/version "0.1.0-SNAPSHOT"}}
> java -jar myapp.jar
Hello, World!
> #2020-12-31 20:01:55 seancorfield (for lib and template types you can clojure -A:jar to build a JAR that should be deployable to http://clojars.org pretty much as-is, with a near-ready-to-roll pom.xml file) #2020-01-02 08:56:58 robert-stuttaford probably an FAQ but my google-fu is lacking ; is there an order of precedence among :mvn/version and :local/root? could i somehow use both, where local is used for dev, but mvn is used for ci? has anything been written about producing this sort of environment specific variance? #2020-01-02 09:27:55 delaguardo We are using special alias for CI builds with :override-deps key inside #2020-01-02 11:07:27 dominicm It'll be an array map, so whichever namespace is found first most likely. As mentioned you should use override deps. #2020-01-02 14:19:46 alexmiller the whole point of stuff like :override-deps is to explicitly support the use case of changing a dep source #2020-01-03 06:35:16 robert-stuttaford thanks folks! #2020-01-03 06:39:11 robert-stuttaford Works beautifully. #2020-01-02 11:00:26 dharrigan Does this -Sdeps DEPS Deps data to use as the final deps file take a map? #2020-01-02 11:01:03 dharrigan i.e., if I want to pass in mvn/repo configuration on the command line, thus ignoring whatever is set in any deps.edn file, is it this switch I require? #2020-01-02 11:08:40 dharrigan yup is the answer 🙂 #2020-01-02 11:16:27 dharrigan Where does deps store downloaded deps, i.e., jars? In the .m2 directory? #2020-01-02 13:20:22 borkdude @dharrigan you can check that with -Spath #2020-01-02 13:20:55 dharrigan ah, thanks #2020-01-02 13:21:00 borkdude This is an example for mvn + git:
src:/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/borkdude/.gitlibs/libs/borkdude/medley/91adfb5da33f8d23f75f0894da1defe567a625c0/src:/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
#2020-01-02 13:21:14 dharrigan yup #2020-01-02 13:21:15 dharrigan thanks #2020-01-02 17:49:54 seancorfield seancorfield/depstar {:mvn/version "0.5.0"} now manages the AOT compilation folder automatically -- no need to manually create classes or add it to the classpath; seancorfield/clj-new {:mvn/version "0.8.3"} updated to use latest depstar (so app template no longer needs classes folder). #2020-01-02 21:06:31 vlaaad And I am still on 0.2.1 because it does what it needs to do 🙂 #2020-01-02 21:28:58 seancorfield That's good to hear! 👍:skin-tone-2: #2020-01-02 21:29:42 seancorfield 0.5.1 and 0.8.4 just released to add support for projects where the main namespace has - in its path and/or its name. #2020-01-07 03:57:29 Nate Sutton how do you run clojure.test tests with a deps.edn file? #2020-01-07 03:58:02 Nate Sutton I feel like I must be missing something #2020-01-07 04:01:02 Nate Sutton oh, it doesn't come with a test runner #2020-01-07 04:01:06 Nate Sutton that'll do it #2020-01-07 04:24:13 andy.fingerhut I wouldn't be surprised if there are better ways, but that is effective and straightforward #2020-01-07 04:25:15 Nate Sutton could I ask by all the commas instead of spaces? #2020-01-07 04:25:30 Nate Sutton I understand that they're considered whitespace in clojure #2020-01-07 04:25:42 Nate Sutton just wondering if that's some kind of limitation of :main-opts or something #2020-01-07 04:31:44 potetm gonna guess the commas are because both ' and " are special in the shell #2020-01-07 04:31:59 potetm so , is a “guaranteed neutral space” #2020-01-07 04:32:14 Nate Sutton but why is space an issue then? #2020-01-07 04:32:31 seancorfield The main opts are round-tripped from shell clojure into files on disk and then back again into the shell and then to java. #2020-01-07 04:33:39 potetm It’s usually safe to assume these kind of shenanigans when storing shell commands. Nobody does it right. #2020-01-07 04:32:52 seancorfield So the quoting/space combinations break in that round-tripping. #2020-01-07 04:32:52 Nate Sutton @seancorfield I did see that but I thought it incomplete because it didn't have a bare example of just using clojure.test itself. turns out there's no test runner #2020-01-07 04:33:08 seancorfield Cognitect's test-runner is in there. #2020-01-07 04:33:20 seancorfield Hence the :test alias and the :runner alias. #2020-01-07 04:33:41 Nate Sutton yep, and I'm using that now. I didn't know there wasn't a built-in test runner in clojure and I was avoiding the additional dependency #2020-01-07 04:34:34 seancorfield Well, you can run tests without the test-runner -- clojure.test does have some basic test running logic -- but you won't have a very nice experience 🙂 #2020-01-07 04:35:10 seancorfield clojure.test/run-all-tests will run all tests that have been loaded (in require'd code). #2020-01-07 04:36:00 Nate Sutton that seems like the part I was missing, because I did try to use clojure.test/run-all-tests as the main function #2020-01-07 04:36:16 Nate Sutton well, that and other pieces #2020-01-07 04:36:29 Nate Sutton I wish there was a note about tests on the deps + cli page on http://clojure.org #2020-01-07 04:47:33 Nate Sutton awesome, thanks! #2020-01-07 04:37:04 seancorfield All the test runners out there look for "test" files (by various means), load the code, and then run all the tests, under whatever filters you provide. #2020-01-07 04:38:15 Nate Sutton for sure, I was just surprised there wasn't one built in, or a note somewhere about there not being one and that you'll have to pick a test runner #2020-01-07 04:38:26 Nate Sutton maybe there is a note somewhere and I didn't find it #2020-01-07 04:38:30 Nate Sutton totally possible ¯\(ツ)/¯ #2020-01-07 04:39:18 seancorfield FWIW, Boot doesn't have a test runner built-in either. Only Leiningen does. #2020-01-07 04:40:11 Nate Sutton ahhh ok, I started with lein #2020-01-07 04:40:30 Nate Sutton still, I imagine it would have been confusing for me even if I hadn't #2020-01-07 04:40:45 seancorfield Most of the test runners out there are additional dependencies that offer more features than Leiningen's built-in stuff... #2020-01-07 04:43:30 Nate Sutton isn't this for a third party tool? #2020-01-07 04:43:34 seancorfield It's under consideration to spin off clojure.test into a Contrib lib for Clojure 1.11. I've volunteered to maintain it and one of the things I'll do is add a bunch of documentation to the project, that covers how to run tests with a variety of tools, and how to filter tests etc. #2020-01-07 04:43:50 Nate Sutton oh, cool #2020-01-07 04:44:06 Nate Sutton well, I have tests running now, so that's nice #2020-01-07 04:44:21 seancorfield Yes, it's for the clojure.test-compatible version of Expectations that I maintain -- but a lot of what's at that link would become "canon" documentation in clojure.test, if I get to maintain it. #2020-01-07 04:51:20 Nate Sutton that looks good #2020-01-07 04:51:29 Nate Sutton the with-test thing is new to me and that'll be super handy #2020-01-07 04:51:50 Nate Sutton thanks, sean #2020-01-07 04:59:25 seancorfield with-test is tricky to use because the default setup for all the test runners will not see those tests -- hence my notes about what needs to be done 🙂 #2020-01-07 05:06:27 Nate Sutton in this case it's in the context of babashka, where a single file with all deps and tests is desirable #2020-01-07 05:07:25 Nate Sutton ahhh spartan.test doesn't seem to include a with-test macro #2020-01-07 05:07:48 Nate Sutton I'm sure it could be added #2020-01-08 08:55:51 plexus @nate_clojurians you might want to look into Kaocha. It was conceived as a test runner for deps.edn that's less spartan than the cognitect runner #2020-01-08 10:30:24 Nate Sutton thanks, I'll check it out :) #2020-01-08 16:36:09 practicalli If your editor has a test runner, eg. CIDER for Emacs, then you only need to ensure the 'test' directory is included in the path. You can add it to :path or as an alias. #2020-01-08 16:40:12 Nate Sutton I do use cider but I'm also running tests in babashka and don't want to work out using a socket repl instead of nrepl with emacs #2020-01-08 17:21:39 plexus Can you elaborate a bit on how you run tests in babashka? What would you expect from a test runner in that environment? #2020-01-08 17:28:10 borkdude We can take this to #babashka #2020-01-10 10:50:52 Mehdi H. Ok never mind, clj -Stree does the trick 🙂 #2020-01-13 07:34:58 slipset > Note that if two dependency versions are at the same depth in the dependency tree, the first declaration wins. #2020-01-13 07:36:16 slipset And the kicker is that wagon-provider-api depends on an older version of plexus-util than the other deps of pomegranate, so by changing the order of the deps in the pom.xml, pomegranate broke. #2020-01-13 07:38:13 slipset Thinking of it, I’m not sure why I’m writing this, I guess it’s just as an amusing story. #2020-01-13 10:48:13 slipset I guess there might be a question in here: Is the order in which the deps are listed in the generated pom.xml predictable in any way? #2020-01-13 10:55:13 dominicm I wouldn't think so. It operates on a hashmap. #2020-01-13 14:01:13 alexmiller no, it's not predictable #2020-01-13 14:02:32 alexmiller you could exclude the plexus-util in whatever branch and then it wouldn't matter, right? #2020-01-13 14:35:52 slipset That's correct, and what I ended up doing. #2020-01-13 17:03:12 avi Does tools.deps.graph support gitlibs? #2020-01-13 17:03:23 alexmiller Yes #2020-01-13 17:06:25 avi Hmm, I see, it looks like maybe not when --size is specified? #2020-01-13 17:06:46 avi which makes sense, but perhaps it would be good to display an error message rather than a stacktrace… #2020-01-13 17:06:55 avi /Users/avi.flax/.gitlibs/libs/expound/expound/9b5778a1a4ed91e2090308a6648ee9072076925a is not a relative path
java.lang.IllegalArgumentException: /Users/avi.flax/.gitlibs/libs/expound/expound/9b5778a1a4ed91e2090308a6648ee9072076925a is not a relative path
at $as_relative_path.invokeStatic(io.clj:414)
at $file.invokeStatic(io.clj:426)
at $file.invoke(io.clj:418)
at clojure.tools.deps.alpha.extensions.git$eval1027$fn__1029.invoke(git.clj:43)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.graph$get_size.invokeStatic(graph.clj:95)
at clojure.tools.deps.graph$get_size.invoke(graph.clj:93)
at clojure.tools.deps.graph$make_dep_node.invokeStatic(graph.clj:107)
at clojure.tools.deps.graph$make_dep_node.invoke(graph.clj:99)
at clojure.tools.deps.graph$make_graph$fn__1522.invoke(graph.clj:125)
#2020-01-13 17:09:18 alexmiller That’s possible, I thought I tested that, but maybe not. It’s supposed to fall back to 0 in that case #2020-01-13 17:10:09 avi Would you like a bug report somewhere? #2020-01-13 17:10:09 alexmiller I’ll take a look when I get a chance #2020-01-13 17:42:54 avi BTW when you do take a look at this, I’m thinking maybe using 0 for gitlibs is a little misleading? It’s more of an unknown… perhaps something like -1 or “unknown” or :man-shrugging: or just leaving it blank…? #2020-01-13 17:10:21 alexmiller Nah #2020-01-13 17:10:28 avi 🆗 thanks2 #2020-01-13 17:10:42 avi BTW reviewing the deps graph for my project is fascinating, and very informative — thank you! #2020-01-13 17:11:38 alexmiller Sorry size isn’t working! #2020-01-13 17:13:34 avi No big deal! #2020-01-13 17:14:15 avi It’s already immediately obvious that I have a single dep bloating my project. It’s a dep I’ve been planning to drop eventually, so now I’m even more motivated to so do! #2020-01-13 17:15:25 avi Here I’ve manually highlighted that dep and all its deps — it’s pretty dramatic! #2020-01-13 17:51:15 alexmiller nice - I'd like to also have an output mode to export the dot file itself instead of the image so you can further mess with it #2020-01-13 17:51:26 avi :+1: #2020-01-13 17:51:49 avi Interested in PRs? #2020-01-13 17:52:26 alexmiller nah, it's part of other changes I want to make #2020-01-13 17:53:41 avi 🤔 it’d be cool if there was a feature to assign a unique background color to each branch from the root — might make it easier to visualize the impact of each root-level dep #2020-01-13 18:00:43 alexmiller you could drop that in a question at https://ask.clojure.org if you want, category of "Contrib libs / tools.deps.graph" and label "request" #2020-01-13 18:01:13 alexmiller worth playing with. I experimented with a variety of color things, never found something that I thought helped, but I didn't try that! #2020-01-13 18:03:20 avi Cool, will do! #2020-01-13 21:27:17 seancorfield @alexmiller A question about tools.deps.graph: it seems that it reads just the current deps file so if you have overrides provided in another deps file, it won't see them? At work we have a "user-level" deps file in the monorepo that "pins" versions of everything via a :defaults alias and then we have {} for deps in each project deps.edn file. #2020-01-13 21:27:36 seancorfield Can I specify multiple deps.edn files to read/merge on the command-line? #2020-01-13 21:29:53 seancorfield (so far everything I've tried in the context of our work repo has either failed outright or produces a blank canvas claiming the size is -1 x -1) #2020-01-13 21:32:47 seancorfield If I generate to a file, I seem to get a graph as expected, so it's perhaps only the default render to a graphics window that fails... #2020-01-13 21:34:17 alexmiller it's the same as clj #2020-01-13 21:34:27 alexmiller in that it uses mostly the identical code #2020-01-13 21:34:44 alexmiller so, no you can't use multiple deps.edn files #2020-01-13 21:35:10 seancorfield Ah, I was confused by the results I was seeing... it does take the user-level deps into account after all? #2020-01-13 21:35:16 alexmiller it should #2020-01-13 21:35:37 alexmiller unless I'm misremembering that #2020-01-13 21:36:15 alexmiller yeah, it does #2020-01-13 21:38:08 seancorfield build [ graph api -a :defaults -o /tmp/graph.png ]
which ends up being
cd api; CLJ_CONFIG=../versions clojure -A:graph:defaults -a :defaults -o /tmp/graph.png
does indeed produce the correct graph for our api subproject. I think getting the blank popup was throwing me off there. #2020-01-14 16:17:27 rschmukler Hey all! I'm consuming a library that seems to have packaged itself as an uberjar including all of its dependencies. I also depend on one of the libraries it uses and it looks like that it's overwriting the requires to an old version (which requires a java class that isn't packaged in the new one). Is there any way for me to force my project to use the new version instead of what it's using? #2020-01-14 16:18:09 rschmukler I'm not super familiar with the internals of how this all works, so if the above diagnosis is complete nonsense, please excuse me! #2020-01-14 16:21:16 alexmiller makes total sense #2020-01-14 16:21:30 alexmiller and no, not easily #2020-01-14 16:21:44 alexmiller you should tell that lib to stop doing that :) #2020-01-14 16:23:39 alexmiller the effect you need is to force your lib version earlier in the classpath ordering. tools.deps mostly does not provide support for explicit ordering #2020-01-14 16:25:50 alexmiller I possibly could come up with a hacky workaround, but it would be a hack. the real solution is the lib #2020-01-17 03:30:11 alexmiller tools.deps.alpha 0.8.264 and clj 1.10.1.496 are now available. #2020-01-17 03:30:17 alexmiller The main change in this is that support for s3 maven repos has been completely replaced and is now using a new impl of the Maven resolver transporter API based on the cognitect aws-api. The old version used the transporter-wagon adapter, the unmaintained spring s3 wagon, and the AWS Java SDK. Those deps have been happily dropped on the floor, and all other deps were bumped as well. Support for some newer AWS credential sources may be better, particularly when running on AWS. Also, you can explicitly set the s3 bucket region in your deps.edn like {:mvn/repos {"my-bucket-repo" {:url ""}}} - this is a feature of this transporter, not a general AWS thing. #2020-01-17 03:31:43 alexmiller The intention here is that any existing use of an s3 maven repo should continue to work as it did before. If you find that not to be the case, please let me know! It has been tested in a number of configurations by several people, but you know how it goes. #2020-01-17 03:33:17 alexmiller tools.deps tool makers may want to do a quick check as this version has some significant transitive dependency differences (hopefully mostly for the better!) #2020-01-17 03:38:33 alexmiller docs have not yet been updated, but I'll do that tomorrow #2020-01-17 04:45:25 tianshu How can I get my clojure cli version?
I usually install with system package manager, It only shows the version of Clojure. #2020-01-17 07:58:16 Michaël Salihi To get the clojure cli verison without launching the REPL as clj -Sverbose do, you can also use clj -Sdescribe.
@alexmillerRight ? #2020-01-17 05:31:58 alexmiller clj -Sverbose #2020-01-17 05:33:35 tianshu @alexmiller thanks! #2020-01-17 07:07:33 slipset I was wondering about the version thing the other day as well, and tried to invoke clj --version which printed the help-text, which I failed to parse (ie I didn’t understand that Print important path info to console meant “and also print version” #2020-01-17 07:07:54 slipset Would it be a nice addition to print the version string in the help-text as well? #2020-01-17 07:21:22 slipset looking through clojure/brew-install this should be simple enough, but is there a Jira project for this project where one might submit a patch? #2020-01-17 14:33:51 alexmiller you can use TDEPS for tickets, but I just added it to the repo #2020-01-17 15:38:16 slipset Thanks 🙂 #2020-01-17 15:38:22 slipset Have a great weekend! #2020-01-17 15:54:30 alexmiller thx, you too #2020-01-17 18:28:52 rschmukler Hey all! I'm looking to publish a library on clojars with a deps.edn file that specifies a :git/url based dependency. I realize that this won't work with a pom.xml - does tools-deps find transitive dependencies using a deps.edn files in addition to pom.xml files? #2020-01-17 19:37:28 alexmiller Yes, both, prefers deps.edn #2020-01-17 19:38:39 alexmiller Or rather will use exactly one but can be either #2020-01-17 18:48:27 borkdude @slipset there's also:
$ clojure -Sdescribe
{:version "1.10.1.496"
,,,} #2020-01-17 19:26:50 seancorfield @borkdude Are any of these differences between deps.clj and the regular clojure script of any concern?
PS C:\Users\sean\clojure\cool> deps -Sdescribe
{:deps-clj-version "0.0.6"
...
:config-files ["deps.edn" "C:\\Users\\sean\\.clojure\\deps.edn" "deps.edn"]
:config-user "C:\\Users\\sean\\.clojure\\deps.edn"
...
:install-dir nil
...
:force ""
:repro ""
...}
PS C:\Users\sean\clojure\cool> clojure -Sdescribe
{...
:config-files ["C:\Users\sean\scoop\modules\ClojureTools\deps.edn" "C:\Users\sean\.clojure\deps.edn" "deps.edn"]
:config-user "C:\Users\sean\.clojure\deps.edn"
...
:install-dir "C:\Users\sean\scoop\modules\ClojureTools"
...
:force False
:repro False
...} #2020-01-17 19:27:43 seancorfield :config-files seems odd with deps.edn in there twice (and :force/`:repro` seem to be "" but that's truthy?) #2020-01-17 19:28:26 seancorfield (I can understand :install-dir being nil since deps.clj is self-contained) #2020-01-17 19:29:09 borkdude yes, that's the reason why install-dir is nil. the rest can be fixed if it's bothering anyone. also the escaping seems different? #2020-01-17 19:30:02 borkdude I think False may be coming from Powershell? it doesn't seem like something cmd.exe would produce #2020-01-17 19:32:57 borkdude should probably read boolean #2020-01-17 19:33:09 seancorfield The macOS/Linux version shows:
:force false
:repro false
#2020-01-17 19:33:20 seancorfield False is just PS's version of false 🙂 #2020-01-17 19:33:53 borkdude I'll make an issue for this. It doesn't seem something that's very urgent, but I'll fix it #2020-01-17 19:34:28 seancorfield I assumed the \\ was due to being produced by Clojure/Java, whereas \ is native PS (or cmd). #2020-01-17 19:34:44 borkdude yeah, this is produced using pr instead of bash #2020-01-17 19:35:07 seancorfield I guess :config-files is no longer used by any tooling so it doesn't matter -- it just looked odd with deps.edn in there twice. #2020-01-17 19:35:42 borkdude yeah, might be indicative of a small bug somewhere. #2020-01-17 19:36:44 borkdude I basically did the naive thing and ported bash straight into clojure without really think hard 🙂 #2020-01-17 19:38:25 seancorfield deps is a great option on Windows -- much appreciated! #2020-01-17 19:39:57 alexmiller Yeah Sean, get off the bench and help out for a change! ;) #2020-01-17 19:40:09 alexmiller I kid, I kid #2020-01-17 19:40:15 borkdude 😄 #2020-01-17 19:40:51 borkdude btw bb.exe is now a thing on Windows so you can execute the deps.clj script as well (so you can tweak it in Notepad++) #2020-01-17 19:41:36 borkdude (I'll now stop the shameless promotion of my tools for the rest of the day) #2020-01-17 19:43:42 rschmukler Looking through the source it doesn't look like transitive deps.edn files are supported in jars... Would a PR be welcome that adds this to the local resolver? similar to how find-pom works currently? #2020-01-17 20:13:51 alexmiller deps.edn in jars is not a thing #2020-01-17 20:14:13 alexmiller and if I wanted to make it a thing, I'd try to design that first #2020-01-17 20:26:44 dominicm What would designing that look like, or do you mean that you would design it? #2020-01-17 20:33:54 alexmiller like designing anything else - starting with goals, thinking about alternatives, examining their tradeoffs, making decisions, implementing #2020-01-17 20:34:06 alexmiller I don't have time to go down that path atm #2020-01-18 05:15:44 seancorfield seancorfield/clj-new {:mvn/version "0.8.5"} -- generated lib and template projects now include :install and :deploy aliases, using @slipset’s excellent deps-deploy, so you can create a JAR clojure -A:jar, and either install locally clojure -A:install or deploy to Clojars clojure -A:deploy; also bumps depstar to 0.5.2 for the NPE bug fix in uberjars with no main class specified. #2020-01-18 05:26:02 seancorfield My dot-clojure/deps.edn file also has these aliases added (although they require a JAR filename as an argument, whereas they're baked into the generated lib and template projects since those already have a name). #2020-01-20 02:52:46 tap Got an error trying to start clj repl with rebl
Error building classpath. org.apache.maven.repository.internal.DefaultModelResolver.<init>(org.eclipse.aether.RepositorySystemSession, org.eclipse.aether.RequestTrace, java.lang.String, org.eclipse.aether.impl.ArtifactResolver, org.eclipse.aether.impl.VersionRangeResolver, org.eclipse.aether.impl.RemoteRepositoryManager, java.util.List)
java.lang.NoSuchMethodException: org.apache.maven.repository.internal.DefaultModelResolver.<init>(org.eclipse.aether.RepositorySystemSession, org.eclipse.aether.RequestTrace, java.lang.String, org.eclipse.aether.impl.ArtifactResolver, org.eclipse.aether.impl.VersionRangeResolver, org.eclipse.aether.impl.RemoteRepositoryManager, java.util.List)
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getConstructor(Class.java:1825)
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(pom.clj:49)
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invoke(pom.clj:39)
at clojure.tools.deps.alpha.extensions.pom$read_model.invokeStatic(pom.clj:60)
at clojure.tools.deps.alpha.extensions.pom$read_model.invoke(pom.clj:54)
at clojure.tools.deps.alpha.extensions.local$eval925$fn__927.invoke(local.clj:68)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at clojure.tools.deps.alpha$expand_deps$fn__1191.invoke(alpha.clj:191)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:190)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:168)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:236)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:216)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:56)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:48)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:82)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:73)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:102)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:96)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:147)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:119)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
deps.edn
{:deps {}
:aliases
{:rebl {:extra-deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/core.async {:mvn/version "0.4.490"}
com.cognitect/rebl {:local/root "/path/to/REBL.jar"}}
:main-opts ["-m" "cognitect.rebl"]}}}
The command I run
clj -R:rebl
Has anyone had the similar issue? #2020-01-20 03:15:57 seancorfield What version does clojure -Sdescribe report? #2020-01-20 03:17:46 seancorfield @tap And you're aware that -R:rebl will add those dependencies but won't run REBL -- it will run the default behavior, which is to start a regular clojure.main REPL. #2020-01-20 03:18:05 tap {:version "1.10.1.496"
:config-files ["/usr/local/Cellar/clojure/1.10.1.496/deps.edn" "/Users/xxx/.clojure/deps.edn" "deps.edn" ]
:config-user "/Users/xxx/.clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/usr/local/Cellar/clojure/1.10.1.496"
:config-dir "/Users/xxx/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
#2020-01-20 03:18:50 seancorfield Your deps.edn file doesn't look valid -- you have five } on that line with com.cognitect/rebl #2020-01-20 03:19:17 tap Sorry #2020-01-20 03:19:18 seancorfield That would put :main-opts outside the whole { } of the deps. #2020-01-20 03:20:50 tap Yeah. Fixed up there. I removed :main-opts just to test it out. Miss that when I readded it to ask question here. #2020-01-20 03:21:50 tap Yes. I'm aware that -R won't run REBL. Just want to make it clearer that it's not related to :main-opts. #2020-01-20 03:22:17 seancorfield Which version of REBL is that local JAR file? #2020-01-20 03:23:15 tap REBL-0.9.220.jar #2020-01-20 03:25:11 seancorfield Ok, I have that setup locally... just a sec... #2020-01-20 03:25:52 seancorfield Yeah, with -R:rebl I get that same error. I've just never run it that way -- that's odd. #2020-01-20 03:26:19 tap I got the same error with -A as well #2020-01-20 03:29:54 seancorfield Let me switch computers and try something... #2020-01-20 03:32:44 seancorfield I tried it with the add-lib branch SHA that I normally use and got the same exception. #2020-01-20 03:33:19 tap Thanks for your help looking into this #2020-01-20 03:35:07 seancorfield Yeah, I can't start my regular dev environment with the latest clojure version with REBL (I've been on vacation for a several days so this is the first time I've tried since the last clojure/`tools.deps.alpha` update) -- /cc @alexmiller looks like the latest release breaks when used with REBL #2020-01-20 03:43:23 seancorfield Looks like clojure isn't set up in a way that earlier releases can be installed with brew 😞 #2020-01-20 03:43:51 seancorfield brew install fails #2020-01-20 03:44:44 sogaiu @seancorfield @tap it looks like i am able to reproduce the problem as well (this is a linux box) #2020-01-20 03:45:35 sogaiu i have an install script for 1.10.1.492 i can try #2020-01-20 03:46:01 seancorfield Yeah, I think the latest changes in clojure/`tools.deps.alpha` are incompatible with the current release of REBL -- because it is AOT'd #2020-01-20 03:46:56 seancorfield Looks like Cognitect will need to release a new version of REBL to fix this (which in turn won't be compatible with earlier clojure/`tools.deps.alpha` builds, I suspect). #2020-01-20 03:47:56 sogaiu i just tried with 1.10.1.492 and it works with the REBL-0.9.220 jar #2020-01-20 03:48:44 sogaiu switching to 1.10.1.496 gives the exception #2020-01-20 03:49:54 seancorfield Weirdly, there's nothing I can see in the REBL JAR or its pom.xml that suggests it wouldn't be compatible... so I guess we have to wait for @alexmiller and/or some other Cognitect folks to chime in tomorrow. Very frustrating we can't downgrade clojure via Homebrew tho'... #2020-01-20 03:52:10 sogaiu indeed #2020-01-20 03:56:53 sogaiu may be the info is dated... #2020-01-20 03:59:41 seancorfield Workaround:
$ curl -O
$ curl -O
$ tar xvfz clojure-tools-1.10.1.492.tar.gz
and then edit your current clojure shell script to point to the clojure-tools/clojure-tools-1.10.1.492.jar from that expansion instead of the installed 496 version. #2020-01-20 04:00:00 seancorfield (at least I'll be back up and running at work tomorrow morning now!) #2020-01-20 04:01:40 alexmiller the exception above from @tap is a regression that is fixed in master and I'll release it tomorrow #2020-01-20 04:02:07 alexmiller you can actually get the older brew formulas if you want to download #2020-01-20 04:04:11 alexmiller rename that to clojure.rb, then brew reinstall --force clojure.rb #2020-01-20 04:04:57 alexmiller brew can only reinstall older versions that you have previously downloaded and installed that you still have in the brew cache #2020-01-20 04:05:19 alexmiller there is no repository of versions like there is with Maven or something, just "the current version" #2020-01-20 04:11:18 seancorfield Is there an option on upgrade or in a specific formula to not delete the old version? #2020-01-20 04:11:45 seancorfield (and good to know on the force install from a downloaded older version) #2020-01-20 04:16:52 alexmiller by default, I think it leaves it #2020-01-20 04:18:52 alexmiller ls ~/Library/_Caches_/Homebrew #2020-01-20 04:19:30 alexmiller and then Formula/clojure under that #2020-01-20 04:19:42 tap Is it appropriate to write about this how to downgrade brew clj package on https://clojure.org under Guides section? I'll do that if y'all think so. #2020-01-20 04:20:56 alexmiller nah, I'd prefer to bury it in reference and link #2020-01-20 04:21:29 tap ok #2020-01-20 04:43:47 seancorfield Under ~/Library/Caches/Homebrew is just the current version of each installed package. #2020-01-20 19:15:47 alexmiller Latest (1.10.1.502) with regression fixes is now available in brew etc #2020-01-20 19:15:59 alexmiller Fixed for pom deps and pom gen #2020-01-20 19:18:10 seancorfield Works with REBL now -- thank you for the swift update! #2020-01-20 19:21:13 alexmiller I also added some tests to catch it earlier next time! #2020-01-20 20:48:26 sogaiu seems to work on a linux here 👍 #2020-01-20 21:25:51 alexmiller oh also, this release has the version as the first line of clj -h #2020-01-20 21:43:23 andy.fingerhut You have been worn down explaining the -Sdescribe and -Sverbose options 🙂 Soon you will succumb to the pull to the dark side of adding a --version option. #2020-01-20 21:58:59 alexmiller even that won't be the end - there will be confusion between the clojure tools version, the clojure tools default clojure version, and the version of clojure specified by the current project :) #2020-01-20 22:03:03 andy.fingerhut Long ago I gave myself slightly more sanity by including a line like the following at the beginning of Eastwood's output, by default: == Eastwood 0.3.6-SNAPSHOT Clojure 1.10.0 JVM 1.8.0_192 == #2020-01-20 22:30:08 seancorfield Heh, that's why all our services report their git tag/SHA, along with the version of Clojure and the JVM at startup! 🙂 #2020-01-20 21:29:24 littleli seems to work on Windows too #2020-01-21 15:54:36 avi Has anyone here ever managed to get Snyk to scan a Clojure project using tools.deps for vulnerabilities? I’m giving it a try and Snyk is helping out but I’m running into Maven problems. #2020-01-21 15:54:45 avi [ERROR] Failed to execute goal on project project: Could not resolve dependencies for project project:project:jar:0.1.0: The following artifacts could not be resolved: com.gfredericks:test.chuck:jar:0.2.10, clj-commons:clj-yaml:jar:0.7.0, hawk:hawk:jar:0.2.11: Could not find artifact com.gfredericks:test.chuck:jar:0.2.10 in central () -> [Help 1]
#2020-01-21 15:55:31 avi But I checked and the pom.xml generated by clojure -Spom already includes both Central and Clojars. #2020-01-21 15:55:55 avi So I’m stuck… #2020-01-21 15:59:18 alexmiller could you post the pom?/ #2020-01-21 15:59:25 avi sure #2020-01-21 16:00:19 avi Just re-running the CI job now to regenerate it. #2020-01-21 16:04:39 avi Doh! picard-facepalm repositories is empty #2020-01-21 16:04:42 avi Never mind! #2020-01-21 16:05:23 avi Or… is there a way to get clojure to populate repositories? #2020-01-21 16:05:38 avi I’m using 1.10.1.483 but could switch to something newer #2020-01-21 16:07:06 alexmiller I'm trying to remember if it's supposed to do that now or not #2020-01-21 16:07:31 avi I’m going to try it with 1.10.1.502 and report back #2020-01-21 16:07:32 alexmiller looks like it is, so might be broken #2020-01-21 16:10:14 avi 😅 #2020-01-21 16:11:04 alexmiller I guess it will only right now add repos explicitly listed in the deps.edn #2020-01-21 16:11:22 avi I’ve got both repos explicitly in deps.edn #2020-01-21 16:12:01 avi I’m testing locally now with 1.10.1.502 #2020-01-21 16:12:24 avi repositories is empty 😅 #2020-01-21 16:12:55 avi my deps.edn has this at the root:
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}} #2020-01-21 16:15:10 alexmiller I'm looking at it #2020-01-21 16:16:16 avi thanks2 #2020-01-21 16:20:08 alexmiller ah, it's subtle but I see the problem #2020-01-21 16:20:24 avi bananadance #2020-01-21 16:25:53 avi any possible workaround? Or maybe I should brush up on my sed skills and try to just patch the pom for now? #2020-01-21 16:25:54 alexmiller for the moment you'll need to manually modify your pom.xml after gen #2020-01-21 16:26:00 avi ah ok #2020-01-21 16:26:05 avi will do — thanks! #2020-01-21 16:26:16 avi Would you like a bug report somewhere? #2020-01-21 16:26:22 alexmiller nah, I've got it #2020-01-21 16:26:39 avi 💪 thanks3 #2020-01-21 17:27:27 alexmiller fixed for next release, but not sure when that will be (and added a test for future) #2020-01-21 17:28:08 avi Fantastic — thanks so much! #2020-01-21 18:59:11 danielcompton Is there a way to get tools.deps to print out all of the :paths and :extra-paths that a given set of aliases resolve to? #2020-01-21 19:00:26 danielcompton -Sdescribe prints out some information, but nothing about the composed config #2020-01-21 19:01:05 alexmiller not currently, although I've been making my way towards making something like that possible #2020-01-21 19:01:38 alexmiller I've wanted something like mvn help:effective-pom so that's been a back burner project of mine #2020-01-21 19:01:50 alexmiller do you care about :paths vs :extra-paths? #2020-01-21 19:02:20 alexmiller or just "all local paths" #2020-01-21 21:45:51 danielcompton All local paths for my usage (send them to clj-kondo) #2020-01-21 21:48:12 alexmiller ok, that's good #2020-01-21 19:37:50 weavejester Does anyone know if there’s an uberjar library for tools.deps that supports merging of files like data_readers.clj? #2020-01-21 19:38:08 ghadi depstar does it #2020-01-21 19:38:12 ghadi maybe? #2020-01-21 19:38:29 ghadi It can be taught to, easily #2020-01-21 19:38:55 weavejester I looked into depstar, but it looks like it has hardcoded that part. I could submit a PR, but I was hoping there was something around already. #2020-01-21 19:39:51 ghadi have you tested it @weavejester ? #2020-01-21 19:40:42 weavejester Yeah, that’s what I mean by it being hard-coded, @ghadi. #2020-01-21 19:40:56 weavejester Notice data_readers.clj is explicitly checked for. #2020-01-21 19:41:16 weavejester And clash-strategy doesn’t look like it can be overridden. #2020-01-21 19:41:28 ghadi oh you meant other files, not that one in particular #2020-01-21 19:41:33 weavejester Yeah #2020-01-21 19:42:30 ghadi not extensible as written #2020-01-21 19:55:27 dominicm @weavejester pack avoids merging altogether. #2020-01-21 19:55:51 dominicm Capsule is the one that works with most of your libraries :) #2020-01-21 20:14:40 seancorfield @weavejester Happy to entertain PRs on depstar to extend the clash handling logic. I inherited that as-is and haven't enhanced that part yet... #2020-01-21 23:12:04 weavejester Are there any disadvantages to Capsule jars? I guess I’m inherently wary of anything that builds on known deployment mechanisms. #2020-01-22 07:28:02 dominicm They use a little bit more memory because they start trampoline jvm, much like lein does #2020-01-22 07:28:31 dominicm Onejar is a little more special, that uses a class loader to do its work #2020-01-22 07:28:54 dominicm You might also want to consider the docker mode, which will directly create a docker image. #2020-01-23 06:26:14 tianshu @U09LZR36F can I package a clojure project into a docker image? #2020-01-23 07:43:53 dominicm Yeah #2020-01-23 08:34:32 tianshu @U09LZR36F can you share your workflow? #2020-01-23 08:38:37 dominicm @U0NBGRGD6 My understanding is that you provide a docker registry and it will upload it there. People are using it with Google. #2020-01-23 11:55:38 rickmoynihan @weavejester: Yes there are definite disadvantages to capsule jars…
Also they extra the jars contents into a cache to build the classpath.
You also get a jvm dance at startup; and you then need to care about proxying various JVM options to the real app vm; i.e. if you want to set the heap etc.
I’ve been meaning to look at onejar as an alternative.
FWIW in spite of the occaisional frustration we currently avoid the issue you’re on about with capsule jars. I think there’s an argument that merging the files has problems too; it would be better to resolve all of them at runtime; i.e. uberjars are really just a hack.
I just wish the JVM platform itself supported this. #2020-01-23 11:57:27 rickmoynihan It might just be better to zip up the classpath and write it in a shell script that starts the jvm tbh. #2020-01-23 11:58:14 rickmoynihan IIRC I wrote this stuff up on a duct issue somewhere… 👀 #2020-01-23 12:02:10 rickmoynihan Also lein-tools-deps and using lein for just the packaging and deployment may also be a solution for you. (Full disclosure: I wrote it) #2020-01-23 14:00:32 dominicm If I had control over how people deployed, I'd tell them to use the skinny mode, aka, dump my project into a folder #2020-01-23 14:01:06 dominicm There's an idea on what you just said though #2020-01-23 14:01:45 dominicm Zip will ignore the prefix leading up to finding the header. So you can have zips which are scripts. #2020-01-23 14:02:46 dominicm Unzip itself to a cache directory, and pass all arguments to the jvm #2020-01-22 04:54:32 Nate Sutton where are git deps cloned to? #2020-01-22 05:00:55 Nate Sutton found it, ~/.gitlibs/libs #2020-01-22 05:01:29 Nate Sutton well, and ~/.gitlibs/_repos #2020-01-22 13:00:20 alexmiller The object dirs and working trees are split there #2020-01-23 10:24:46 sveri Hi, is there a way to define an alias that extends an existing alias? #2020-01-23 10:30:16 vlaaad no alias composition, as far as I know #2020-01-23 10:34:14 sveri Ok, thank you 🙂 #2020-01-23 20:49:04 sam Are private git repos supported in deps.edn? The documentation says public and private are supported, but I'm not sure how to provide credentials for private and I can't find any instructions. #2020-01-23 20:55:07 alexmiller yes and yes #2020-01-23 20:55:27 alexmiller it can be a bit fiddly though, so when it doesn't work, come back here :) #2020-01-23 20:58:04 sam Thanks! #2020-01-23 21:09:55 sam I got the private repo part to work, but got this error:
Error building classpath. Manifest type not detected when finding deps for provider-data-pipeline/provider-data-pipeline
which is apparently due to the fact that that the target repo is using leiningen for dependency management instead of deps.edn. Is that right? #2020-01-23 21:14:35 seancorfield @samwagg0583 Correct. t.d.a can only process deps.edn and pom.xml based git dependencies. #2020-01-23 21:16:03 borkdude if it's a dependency that only has sources and no other deps in its project.clj you can declare a manifest type:
Optional key :deps/manifest
Specifies the project manifest type
Default is to auto-detect the project type (currently either :deps or :pom)
This works for e.g. libs like medley #2020-01-24 02:25:34 avi I like this tip! Thanks! #2020-01-23 21:19:24 seancorfield Yes, true, you can "force" :deps and it treats it as if deps.edn exists with {} as its contents. #2020-01-23 21:30:16 sam Thanks everyone. #2020-01-23 21:54:38 sam What about using lein pom to generate a pom and committing that to the target git repo? I do have control over the target, but I might not be able to convert it to deps.edn, at least in the short term. Curious if you guys have heard of anyone using that pattern and how it's worked out. #2020-01-23 21:56:08 sam I mean, I'd rather not have 2 different listings of deps like that. But it seems like an option. #2020-01-23 22:09:11 seancorfield @samwagg0583 If the repo has dependencies, then adding a pom.xml file seems like a good compromise. #2020-01-23 22:09:41 seancorfield Just bear in mind that if someone updates project.clj and forgets to update pom.xml you'll have inconsistent dependencies downstream. #2020-01-23 22:10:24 seancorfield Another alternative would be to put the basic dependencies from project.clj into deps.edn in that repo and use a plugin to pull them in from that file... #2020-01-23 22:11:21 seancorfield So then deps.edn would be the "system of record" for the primary dependencies and both the Lein project and downstream consumers via git will see the same file, same deps. #2020-01-28 22:09:45 sam @U06HHF230 Thanks for the info! #2020-01-28 22:12:28 sam For the moment, I went the route of publishing a pom.xml in the target repo. But I may explore your lib more in the future. #2020-01-24 14:22:52 holyjak Hello! Does anyone have an example how to create a simple clojure + java based project with deps.edn? (Clojure will use some Java classes). Quick googling revealed nothing relevant... #2020-01-24 14:32:23 alexmiller deps does not explicitly support Java compilation #2020-01-24 14:33:03 alexmiller so you either need something that can combine deps with javac, or split the project into java part and clojure part #2020-01-24 15:04:47 sogaiu @holyjak if you haven't checked out @mike1452's recent 2 announcements for the clj-new templates slib and sapp, may be it's worth a look? #2020-01-24 16:04:12 holyjak Thank you! #2020-01-24 16:02:08 alexmiller moving here from announcements sub-thread. @seancorfield I'm seeing an error trying to use clj-new with that sapp template: #2020-01-24 16:02:12 alexmiller clj -A:new qwerty/asd
Failed with: org.apache.maven.repository.internal.DefaultModelResolver.<init>(org.eclipse.aether.RepositorySystemSession,org.eclipse.aether.RequestTrace,java.lang.String,org.eclipse.aether.impl.ArtifactResolver,org.eclipse.aether.impl.VersionRangeResolver,org.eclipse.aether.impl.RemoteRepositoryManager,java.util.List)
Execution error (ExceptionInfo) at clj-new.helpers/resolve-remote-template (helpers.clj:149).
Could not load artifact for template: slib
Tried coordinates:
[slib/boot-template "RELEASE"]
[slib/lein-template "RELEASE"]
Full report at:
/var/folders/2k/2l7ch1xn7lb5gj86tgp1ygww0000gn/T/clojure-13105343161273067757.edn
#2020-01-24 16:03:13 alexmiller The DefaultModelResolver thing is either the problem I just fixed in latest tools.deps.alpha or a version conflict due to pomegranate (both kind of looked similar errors). I notice that clj-new is using a pretty old version of tools.deps? #2020-01-24 16:06:24 mike_ananev hmm, interesting. i just checked out template on clear centos 7 with latest clj-new 0.8.5 and tools deps. everything works fine. can't reproduce DefaultModelResolver error. #2020-01-24 16:06:35 alexmiller My clj-new alias is
:new {:extra-deps {seancorfield/clj-new {:mvn/version "0.8.5"}}
:main-opts ["-m" "clj-new.create"]} #2020-01-24 16:07:35 mike_ananev so, my alias looks the same:
:new {:extra-deps {seancorfield/clj-new {:mvn/version "0.8.5"}}
:main-opts ["-m" "clj-new.create"]} #2020-01-24 16:08:17 alexmiller if it's the conflict with transitive pomegranate maven deps, that may be the kind of thing that is arbitrary and resolves differently on different machines. can you dump me clj -Stree -A:new ? #2020-01-24 16:08:24 alexmiller so I can compare #2020-01-24 16:09:02 alexmiller and also, what version of clj are you on (`clj -Sdescribe`) #2020-01-24 16:10:41 mike_ananev {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "75181c1e10351817054547"}, :content ("[email protected]")} #2020-01-24 16:16:11 alexmiller that was same for me #2020-01-24 16:16:27 alexmiller bumping to latest tools.deps with clj-new worked #2020-01-24 16:16:30 alexmiller :new {:deps {seancorfield/clj-new {:mvn/version "0.8.5"}
org.clojure/tools.deps.alpha {:mvn/version "0.8.640"}}
:main-opts ["-m" "clj-new.create"]}` #2020-01-24 16:17:14 alexmiller probably because I'm not calling the DefaultModelResolver at all in latest #2020-01-24 16:17:34 alexmiller I guess @seancorfield that I would suggest maybe bumping tda in clj-new #2020-01-24 16:20:55 mike_ananev I will add org.clojure/tools.deps.alpha {:mvn/version "0.8.640"} as recommendation to README in case of errors #2020-01-24 16:22:33 alexmiller well, I'd hold off on that, I hate to cargo cult workarounds like that #2020-01-24 16:22:45 alexmiller I could file an issue? #2020-01-24 16:23:52 alexmiller I work with a hacked up clj environment a lot of the time and this may truly be just something on my machine (although I think I have undone all of that) #2020-01-24 16:27:28 seancorfield @alexmiller Can you open an issue on clj-new to bump the deps? That way I won't forget when I get to work, and I'll cut 0.8.6 today. #2020-01-24 16:27:36 alexmiller sure #2020-01-24 16:32:28 alexmiller done #2020-01-24 17:05:54 seancorfield @mike1452 seancorfield/clj-new {:mvn/version "0.8.6"} is up on Clojars -- can you try that out (without adding t.d.a.) and see if it resolves your issues? #2020-01-24 17:06:09 seancorfield Also @alexmiller 🙂 #2020-01-24 17:07:57 seancorfield I also bumped Pomegranate to the latest version, which is on clj-commons now, which may (or may not) cause new problems since the group ID has changed and code will possibly get two versions of it... #2020-01-24 17:09:04 seancorfield (the Jackson dependency hell conflicts with various templates are why 2.7.5 of that is also pinned in clj-new since Luminus' template was breaking before -- due to it relying on Leiningen's version resolution avoiding a conflict there) #2020-01-24 17:24:45 alexmiller btw, as of the new s3 transporter in tda, it no longer has a jackson dep #2020-01-24 17:33:23 seancorfield OK, I may remove that and see if Luminus or other project templates still break. #2020-01-24 17:34:51 alexmiller you may still be pulling it in from multiple places, but tda is not one of them :) #2020-01-24 17:36:03 seancorfield Luminus templates seem to generate OK without clj-new's deps on Jackson so in 0.8.7 I'll take them out. #2020-01-24 17:36:13 alexmiller also, you probably want the clj-new alias to be :deps, not :extra-deps so that you override, instead of add to, the deps of the current project deps.edn if there is one #2020-01-24 17:36:36 seancorfield Good point. That wasn't available before. #2020-01-24 17:36:40 alexmiller usually it probably doesn't matter as you're making a new project #2020-01-24 17:36:59 alexmiller (but note that this requires a sufficiently new clj version, and you'll get an error if using older) #2020-01-24 17:37:59 alexmiller I'd recommend at least clj 1.10.1.483 (Nov 2019 vintage) #2020-01-24 17:38:14 alexmiller so maybe that's something to hold off on #2020-01-24 17:41:16 seancorfield Yeah... I'll give it a month or two and a few more releases of clj before I make that change. I just saw someone still using clj-new 0.5.5 😐 #2020-01-24 17:16:40 mike_ananev @seancorfield I'll check. just a moment. #2020-01-24 17:22:35 alexmiller @seancorfield worked #2020-01-24 17:23:40 defa I have a tools-deps based project with a Clojure backend (lacinia-pedestal) und a re-frame frontend (ClojureScript). So it is a mixed Clojure/ClojureScript project. I’m using :aliases to run the server and the frontend (via shadow-cljs). Am I supposed to define the deps as :extra-deps in the related aliases? I just debugged an issue where jetty stopped working with very strange exceptions and that was related to having pedestal-lacinia and re-graph in :deps which caused incompatible versions of jetty libraries loaded. Moving the re-graph dep into the frontend alias solved the problem. #2020-01-24 17:25:58 alexmiller if you have deps that are only needed when the alias is enabled, then they should be :extra-deps in that alias #2020-01-24 17:26:07 alexmiller sorry if I misinterpreted the question #2020-01-24 17:28:22 defa I guess that’s the answer, since it is working. Is it in general a good/bad idea to have client and server in one deps.edn? #2020-01-24 17:30:35 alexmiller I don't think it's either good or bad #2020-01-24 18:27:15 dominicm True neutral? #2020-01-24 18:28:30 alexmiller 💯 #2020-01-24 18:28:44 alexmiller or maybe lawful neutral #2020-01-24 18:51:04 defa At least it is very convenient when working with cursive inde IntelliJ IDEA. Guess I really have to give emacs a try one day… #2020-01-24 21:12:28 dominicm Fwiw, I find it equally useful with vim, and my colleagues do with emacs. #2020-01-24 21:13:27 dominicm I try to defer splitting projects as long as possible. Development is faster when you don't have to fabricate arbitrary boundaries between things where you haven't found the boundary yet #2020-01-25 17:35:39 sveri While I agree with this sentiment in general, dealing with dependencies will cause troubles sooner or later (not only in clojure / java, but development in general).
If I have learned something over the decades it's these rules.
1. Uses as few dependencies as possible.
2. Separate these as much as possible.
3. Use the same version of a dependency everywhere, if you can.
4. Avoid gradle and eclipse, as eclipse still cannot separate classpathes like gradle does. #2020-01-25 18:28:38 dominicm Tools deps has made my life significantly better in this space, so has the clojure culture of not breaking the api #2020-01-24 17:40:01 mike_ananev @seancorfield @alexmiller I have updated templates on GitHub with clj-new 0.8.6+ #2020-01-27 22:56:03 salam i hope this is the right channel to ask questions about deps.edn and the clojure cli tools.
i was wondering whether deps.edn can be spec'ed so that if one makes a mistake (e.g., putting a string literal where a symbol is expected), the cli tools can give some kind of feedback on the said mistake as spec errors. has the core team considered this? #2020-01-27 23:30:10 andy.fingerhut This is a reasonable place to ask such questions about those tools. I'm not part of the core team, and do not know their views on stricter checking of deps.edn file contents. #2020-01-27 23:49:19 salam thanks for the response, @U0CMVHBL2 🙂 #2020-01-27 23:39:44 alexmiller There are specs in the tools.deps repo #2020-01-27 23:40:18 alexmiller It’s not currently checked as we were trying to support older Clojure versions although we could probably bump that req now #2020-01-28 00:01:04 alexmiller Yeah, I used that to validate all the deps.edn I had lying around :) #2020-01-28 22:55:33 alexmiller so I kind of got distracted into implementing parallel downloads for tools.deps today #2020-01-28 22:56:09 alexmiller might need some hearty volunteers to try it before I release it though #2020-01-28 22:57:27 alexmiller it was one of those things where I got it working in about 20 minutes, then spent 3 hrs properly rewriting it to account for error handling and all the other real work #2020-01-28 22:57:56 alexmiller but it's definitely faster when you're downloading a bunch of junk #2020-01-28 23:00:36 alexmiller it all of a sudden got a lot slower a while ago and I thought I had broken something until I realized my son was downloading a giant game 😄 #2020-01-28 23:32:37 seancorfield How should we test it for you? #2020-01-28 23:44:43 alexmiller well, I'm still hacking on it a bit, but I'll try to make something available #2020-01-29 04:10:08 dharrigan Happy to volunteer too! #2020-01-29 04:48:50 alexmiller the interesting bit is that parallelizing the jar downloads only got me so far because with Clojure artifacts, the pom is of significant size compared to the jar files and we end up downloading a lot more metadata during version selection #2020-01-29 04:50:02 alexmiller so, working through parallelizing the metadata download too, which is a lot harder to work into the algorithm #2020-01-29 04:50:37 alexmiller but, still seems doable #2020-01-29 07:50:26 dominicm Very cool. Does maven do parallel at all? #2020-01-29 07:50:40 alexmiller yeah #2020-01-29 07:51:01 alexmiller I have parallel metadata during expansion working now #2020-01-29 07:51:42 alexmiller the benefits vary a lot depending on how much metadata/how much data/how much parallelism/how bushy the dep tree is, etc #2020-01-29 07:52:10 alexmiller for something like the luminus template, for lein to download the full dep tree is ~43 sec #2020-01-29 07:52:19 alexmiller clj before was 73 sec and is now 10 sec #2020-01-29 07:52:52 alexmiller that's kind of an extreme (lots of jars, huge fanout, lots of java, so big jars) etc #2020-01-29 07:53:30 alexmiller but 20-60% improvement is a common range #2020-01-29 07:54:19 alexmiller most of the time of course you're not really seeing those full numbers because you already have some or most of the libs downloaded in the cache #2020-01-29 09:15:07 p-himik It seems that "RELEASE" and "LATEST" are not available in Maven 3, but tools.deps seem to support them.
However, I cannot find any mention of these aliases in the documentation.
What's the deal with them? Can people rely on them? Can developers safely recommend using them for e.g. such things as tools in the user's deps.edn?
Also, I know that "RELEASE" is susceptible to cache issues - if no new dependency is needed to be retrieved during dependency resolution, the latest release available locally will be used (i.e., no check for a new release will be made) or something like that.
Is "LATEST" also susceptible to such issues? Or does it reliably make a check for a new version each time the dependency resolution is done? #2020-01-29 13:26:26 alexmiller They are not supported and break the cache checking #2020-01-29 13:27:21 alexmiller That said, they do actually work and can be useful for one off calls in something like -Sdeps #2020-01-29 13:28:08 alexmiller I recommend using real versions instead #2020-01-29 13:30:41 p-himik @thheller ^ FYI
Yeah, I use "RELEASE" only for clj-kondo, tools.deps.graph, etc. Something that runs only to facilitate the development. #2020-01-29 15:01:41 alexmiller note that if you use RELEASE for tools, you generally won't get new versions of the tools after you first run them, unless you -Sforce (because the hashed key is just seeing "RELEASE" every time) #2020-01-29 17:18:58 seancorfield Good to know. Explains some of the slightly odd behavior I thought I was seeing from time to time since my dot-clojure deps.edn uses RELEASE a lot. #2020-01-29 17:19:39 seancorfield clj-new has moved away from that and tracks explicit versions now. #2020-01-29 17:20:32 seancorfield It would be nice if there was some way in deps.edn to flag a dependency that should be rechecked periodically rather than cached indefinitely. #2020-01-29 17:24:18 alexmiller The issue is that the cache check is done in bash to avoid jvm load so the hash is purely text based #2020-01-29 17:25:51 alexmiller So there is no easy way to do what you’re suggesting without a lot of trade offs #2020-01-29 17:27:38 seancorfield Yeah, I think it would need an additional file in .cpcache for "dependencies that should be rechecked" which is probably more complexity than anyone would want for the occasional use of this feature. #2020-01-29 17:31:13 seancorfield Is it worth me spending time thinking about this problem or would such a proposal not be considered anyway? #2020-01-29 18:43:44 alexmiller Way down my list #2020-01-29 18:46:27 seancorfield I'll take that as a "no" then 🙂 #2020-01-29 15:02:52 alexmiller probably you're changing your local deps.edn or clj install frequently enough that you are periodically re-caching and don't notice this #2020-01-29 15:21:48 borkdude it would be cool if tools.deps could resolve RELEASE in addition to tags maybe? #2020-01-29 15:22:12 borkdude as in replace RELEASE with the most recent version in your deps.edn file #2020-01-29 15:35:53 alexmiller I'm not planning on adding any more mucking with deps.edn #2020-01-29 15:36:14 alexmiller but does not need to be in tools.deps - this itself could be a tool #2020-01-29 15:36:39 alexmiller I'm not sure that's actually the best solution though #2020-01-29 15:38:36 dharrigan https://github.com/Olical/depot does something similar, although I do not know if it processes "RELEASE" versions, but it updates the deps.edn in place definitely for deps that have versions. #2020-01-29 15:44:41 dharrigan Maybe it does Besides SNAPSHOT versions --resolve-virtual will also handle the special version strings "RELEASE" and "LATEST" #2020-01-29 16:23:53 robert-stuttaford can i remove a dependency in an override? #2020-01-29 16:24:53 alexmiller iirc, no, but this has been on my radar for a while #2020-01-29 16:25:23 robert-stuttaford I have datomic-pro overriding my tools.reader dep #2020-01-29 16:25:34 robert-stuttaford looks like i'll have to use a cljs alias #2020-01-29 16:26:09 thheller not exactly overriding. it contains a AOT compiled version of tools.reader which clojure will load instead of the actual tools.reader version on the classpath #2020-01-29 16:26:18 alexmiller that's been fixed in datomic-pro, but not sure if that's out yet #2020-01-29 16:29:46 alexmiller yeah that fix is in 0.9.6024 #2020-01-29 16:41:00 robert-stuttaford thanks, @alexmiller! #2020-01-29 16:24:33 rickmoynihan @alexmiller: I vaguely remember you once mentioning a tools.deps top level keyword for tooling to use that wouldn’t pull in the main :deps? I can’t see any mention of it, so I’m guessing it’s not implemented yet? #2020-01-29 16:25:26 borkdude you actually can remove a dependency with the classpath overrides #2020-01-29 16:25:35 alexmiller if you are in an alias, you can use :deps to replace instead of :extra-deps to add #2020-01-29 16:25:46 rickmoynihan ahh that was it, thanks. #2020-01-29 16:26:55 borkdude ^ setting the classpath override to nil will remove it from the classpath #2020-01-29 16:27:12 robert-stuttaford oh man thank you #2020-01-29 16:27:28 borkdude only works in an alias I think #2020-01-29 16:28:35 thheller or you could have a server/clj alias so cljs alias doesn't inherit all the other deps as well #2020-01-29 16:31:39 robert-stuttaford @borkdude's suggestion worked! :cljs {:classpath-overrides {com.datomic/datomic-pro nil}} #2020-01-30 10:17:37 onetom i have some issues suddenly downloading the expectations/clojure-test dependency:
% clj -Aexpect
Downloading: expectation/clojure-test/1.2.1/clojure-test-1.2.1.pom from
Downloading: expectation/clojure-test/1.2.1/clojure-test-1.2.1.jar from
Error building classpath. Could not find artifact expectation:clojure-test:jar:1.2.1 in central ()
my project's deps.edn explicitly specify these repos:
:mvn/repos
{"clojars" {:url ""}
"central" {:url
""
#_""}
"" {:url ""}
"datomic-cloud" {:url ""}}
im not sure how to debug the situation #2020-01-30 13:00:54 practicalli The #tools-deps channel is also useful for deps.edn issues #2020-01-30 13:01:41 practicalli You can also send me a direct message if you need help and cant find where to get help #2020-01-30 10:19:24 borkdude Maybe you have a typo, expectation vs expectations? #2020-01-30 10:19:26 onetom my ~/.clojure/deps.edn also contains one repo:
:mvn/repos {
"datomic-cloud" {:url ""}
}
how are this per-user and per-project :mvn/repos are getting merged? #2020-01-30 13:52:52 alexmiller Just that - they are merged, project merged over user merged over built in #2020-01-30 10:20:16 onetom indeed, thanks @borkdude! #2020-01-30 10:22:11 onetom i tried to copy out the deps specification from a remote, terminal based emacs, running in an iterm2, but it has mouse support, so i could only yank it into the remote emacs' kill ring...
that's why i retyped it locally... incorrectly... #2020-01-30 10:25:01 onetom im battling with emacs for months now, slowly learning more and more about it but i still end up falling back to cursive to get things done...
im really wondering if there is some easy way to get into clojure development with emacs or everyone goes through these serious birthing pains?
i found https://practicalli.github.io/clojure/ recently which seems promising #2020-01-30 10:28:08 borkdude @onetom This is a bit off topic for this channel, but feel free to continue about learning clojure in #beginners #2020-01-30 10:28:39 borkdude (or #emacs, #off-topic , etc) #2020-01-30 14:29:07 dharrigan that graph view for deps is aaaammmmaaaaazing 🙂 #2020-01-30 14:44:05 maleghast graph view..? I think that I may have missed something, but scrolling back I can't find it... 😞 #2020-01-30 15:22:15 dominicm Tools.deps.graph #2020-01-30 15:22:19 dominicm On github #2020-01-30 14:29:11 dharrigan Thanks Alex! #2020-01-30 14:32:57 alexmiller try it with --size too #2020-01-30 15:30:30 kszabo Sadly --size doesn’t work with gitlibs based monorepo dep that uses :deps/root specified deps.edn which references a sibling monorepo folder dep via :local/root "../aws" . Quite the contrived example. Here’s the obfuscated stacktrace:
/Users/kszabo/.gitlibs/libs/monorepo/aws/8badc505955040c6a8780bfab4e0484ca4c69a45/aws is not a relative path
java.lang.IllegalArgumentException: /Users/kszabo/.gitlibs/libs/monorepo/aws/8badc505955040c6a8780bfab4e0484ca4c69a45/aws is not a relative path
at $as_relative_path.invokeStatic(io.clj:414)
at $file.invokeStatic(io.clj:426)
at $file.invoke(io.clj:418)
at clojure.tools.deps.alpha.extensions.git$eval1027$fn__1029.invoke(git.clj:43)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.graph$get_size.invokeStatic(graph.clj:95)
at clojure.tools.deps.graph$get_size.invoke(graph.clj:93)
at clojure.tools.deps.graph$make_dep_node.invokeStatic(graph.clj:107)
at clojure.tools.deps.graph$make_dep_node.invoke(graph.clj:99)
at clojure.tools.deps.graph$make_graph$fn__1522.invoke(graph.clj:125)
#2020-01-30 15:44:10 alexmiller yeah, that came up recently, will fix on next rev #2020-01-30 21:08:18 alexmiller is now fixed in tools.deps.graph 0.2.30 #2020-01-30 16:03:48 dominicm If I have a main that takes arguments that look like clojure's (e.g. pack) and I run like so:
clojure -A:pack -- -A:for-pack-pls
Then both the -- and the -A:for-pack-pls are passed to pack, but I'd only expect the -A:for-pack-pls to be passed. #2020-01-30 16:06:53 dominicm Oh, looks like it's because -- is an unknown flag to the cli, so it just passes it along... It would be handy if it actually followed the -- convention itself. #2020-01-30 16:07:48 dominicm Open to a patch? #2020-01-30 16:10:41 borkdude I was also considering this for the irrelevant unclojure. #2020-01-30 16:12:25 dominicm What's unclojure? 😄 #2020-01-30 16:13:56 borkdude you know, the off-topic one 🙂 #2020-01-30 16:14:53 dominicm Haha, yes. #2020-01-30 16:19:20 alexmiller is that a question for me re tools.deps or for pack? #2020-01-30 16:19:27 borkdude tools.deps #2020-01-30 16:19:42 alexmiller sorry, I don't understand what's being requesteed #2020-01-30 16:19:53 alexmiller what is the -- convention? #2020-01-30 16:20:02 borkdude the arguments after -- should be passed as *command-line-args* #2020-01-30 16:20:14 borkdude s/should/could #2020-01-30 16:20:23 alexmiller so is this really a clojure.main request? #2020-01-30 16:20:27 dominicm @alexmiller for you. #2020-01-30 16:20:38 dominicm No, this is a tools.deps request. The patch would be to the clojure script #2020-01-30 16:20:50 borkdude the only tool where I've seen this is the docker cli #2020-01-30 16:20:56 borkdude but I bet there are many others #2020-01-30 16:21:13 dominicm git uses it too, in case you have files starting with - #2020-01-30 16:21:43 dominicm diff --git a/src/main/resources/clojure b/src/main/resources/clojure
index 851d1a6..c6170a3 100755
--- a/src/main/resources/clojure
+++ b/src/main/resources/clojure
@@ -55,6 +55,10 @@ do
help=true
shift
;;
+ --)
+ shift
+ break
+ ;;
*)
break
;;
Code is easier to describe, here's the change I'm proposing ^ #2020-01-30 16:23:36 alexmiller just restating to make sure I'm getting it...
clojure takes command lines of the form clojure [dep-opt*] [init-opt*] [main-opt] [arg*] and makes java command lines of the form java [java-opt*] -cp classpath clojure.main [init-opt*] [main-opt] [arg*] and the -- introduces a separator between [dep-opt*] and the rest? #2020-01-30 16:24:25 dominicm Yes (Although I'm now going to double check that's what happens with my patch) #2020-01-30 16:25:06 alexmiller not clear to me how that patch does that, but I get the intent now #2020-01-30 16:26:19 dominicm *) delegates to init-opt by catching unrecognized arguments (e.g. -e) and passing them on. -- works similarly, but it does a shift before break in order to remove the -- before continuing. #2020-01-30 16:27:15 alexmiller ok, I got it #2020-01-30 16:27:38 alexmiller seems ok to me if you want to make a ticket/patch #2020-01-30 16:27:45 dominicm haha, I have a silly way to prove that it works.
❯ echo '(println "HELLO")' > -A:kafka
❯ clojure -- -A:kafka
HELLO
#2020-01-30 16:27:51 dominicm Okay, sending both in now 🙂 #2020-01-30 16:27:58 alexmiller also need to handle: windows script, help text, and man file #2020-01-30 16:29:16 dominicm Maybe a stupid question, where's the windows script? Not in brew-install? #2020-01-30 16:30:19 dominicm I don't see it on master #2020-01-30 16:30:54 alexmiller that is the correct branch #2020-01-30 16:30:57 alexmiller 1.10.1 #2020-01-30 16:31:24 borkdude btw dominic, Powershell also works on Linux and Mac nowadays, if you want to test some snippets #2020-01-30 16:31:39 alexmiller the change is probably pretty obvious #2020-01-30 16:32:18 borkdude even then it's nice to test, unless you're not making obvious mistakes all the time like I do 😛 #2020-01-30 16:32:23 dominicm My distro hasn't got powershell landed yet, some bug with libressl and dotnet. #2020-01-30 16:32:31 alexmiller the man file is in doc/clojure.1 #2020-01-30 16:34:53 alexmiller another test to check is that there isn't an issue with the built-in clojure.main support for - arg #2020-01-30 16:35:29 alexmiller I think it's fine, but would be good to check it #2020-01-30 16:36:37 dominicm There isn't a problem 🙂 #2020-01-30 16:41:07 dominicm @alexmiller Just to check, happy with "Stop parsing arguments, and pass remaining arguments to clojure.main" as the doc? #2020-01-30 16:52:57 dominicm I've gone ahead with that string, can easily swap it out if you'd prefer something else 🙂 #2020-01-30 17:02:51 alexmiller I don't know, whatever. I'll update after I apply and look at it #2020-01-30 17:04:35 dominicm haha, okay 🙂 #2020-01-30 17:17:51 jffry aws-vault uses this pattern (a tool for acquiring temporary credentials in AWS, and then running a command with those creds as env vars)
aws-vault exec profile-name --flag-for-aws-vault -- some-command --flag-for-command #2020-01-30 18:37:49 seancorfield "The argument ‘--’ terminates all options; any following arguments are treated as non-option arguments, even if they begin with a hyphen." (to save folks reading it) #2020-01-30 18:51:01 alexmiller thx #2020-01-30 18:55:33 borkdude @seancorfield I looked for that on the tools.cli README earlier because of this conversation but Cmd+f on -- didn't get me anything #2020-01-30 19:01:54 seancorfield @borkdude Yeah, the README doesn't mention it -- it just links to the GNU doc as the "spec" of the behavior. #2020-01-30 19:02:25 borkdude Ah -- thanks 🙂 #2020-01-31 04:08:18 alexmiller tools.deps.alpha 0.8.661 and clj 1.10.1.507 are now available
• New: concurrent downloads make things significantly faster when downloading lots of stuff
• New: -Sthreads can be used to explicitly set the concurrency level, but you generally shouldn't need it
• New: -- can be used in clj as a separator between clj dep args and clojure.main args. For example clj -A:tool -- -A:foo will activate the tool alias, which may specify a main class, then -A:foo will be passed to that program (thanks Dominic Monroe!)
• Fix: regression in -Spom not populating <repositories> since tda 0.8.573
• Fix: manifest type lookup on resolved git dep coordinates (probably of interest to almost no one, but affected tools.deps.graph) #2020-01-31 05:06:35 seancorfield Nice job! The concurrent downloads is very cool! #2020-01-31 05:07:08 seancorfield brew doesn't known about 507 yet it seems... caching? #2020-01-31 05:08:16 seancorfield Ah, interesting. It knows about it on macOS but not on Linux yet. #2020-01-31 05:16:06 andy.fingerhut Not familiar enough with brew yet, but perhaps brew update will force it to get fresh version info for all packages. #2020-01-31 05:24:21 seancorfield brew upgrade clojure causes brew to update all its formulae anyway I think... certainly looked like it did. #2020-01-31 05:24:57 seancorfield I suspect the Linux archives lag behind the macOS archives. #2020-01-31 05:27:43 seancorfield {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "661503070805262223352d3229364b373354332c5728"}, :content ("[email protected]")}
Ubuntu on WSL1 on Windows 10. #2020-01-31 05:30:33 andy.fingerhut Probably. I get same result on Linux vs. macOS right now. #2020-01-31 05:33:35 alexmiller Linux always lags #2020-01-31 10:07:10 vlaaad I haven't really used deps.clj, but I think it's great because it has good windows support — through cmd, not power-shell #2020-01-31 10:08:23 vlaaad I wonder how probable it is for clojure/`clj` tools to start using graalvm for cross-platform support #2020-01-31 10:09:41 borkdude if cognitects wants to adopt (parts of) deps.clj, it's free and I'm there if they need help #2020-01-31 13:53:33 alexmiller I'm somewhat interested in exploring graal for the classpath construction phase of clj, however, I do not think it makes sense for the official clojure tool to run something other than the jvm on windows - this is how people start their actual Clojure apps, and Graal is not an officially supported platform for Clojure. clj is possibly a different story for interactive repl use, but even then I don't think it makes sense to be running a non-official Clojure interpreter as the repl. #2020-01-31 13:55:21 alexmiller fortunately, there's no reason we can't have multiple options! #2020-01-31 14:03:14 borkdude @alexmiller I think you're misunderstanding something. deps.clj is only a replacement for the bash part. it's compiled to native using the official clojure and doesn't use a non-official clojure interpreter. #2020-01-31 14:03:34 borkdude like the bash script, it invokes a JVM #2020-01-31 14:04:02 borkdude there also an option to execute the script (non-binary) with babashka, but you don't have to #2020-01-31 14:04:14 alexmiller ah, so that is different than what I understood #2020-01-31 14:05:01 alexmiller I still think it's unlikely Rich would ok using a separate interpreter in this stack, but I could be wrong #2020-01-31 14:05:28 borkdude if you're using the natively compiled deps.exe you are *not* using a separate interpreter #2020-01-31 14:06:17 borkdude deps.clj can run as native (compiled with GraalVM) *or* as an interpreted script (interpreted by babashka which is itself compiled with GraalVM) #2020-01-31 14:06:33 alexmiller neither of those are supported Clojure things though #2020-01-31 14:06:37 vlaaad Yeah, deps.clj is a native windows binary #2020-01-31 14:07:00 vlaaad so deps.exe really #2020-01-31 14:07:42 borkdude yeah, that's why I chose the name deps.exe to distinguish from the script deps.clj (which is really just a Clojure source file which you can run on the JVM, or with babashka, it's portable) #2020-01-31 14:08:01 borkdude but I get the message and I'm not trying to sell it to you 😉 #2020-01-31 14:08:24 alexmiller gotcha, the Graal path is something that could potentially be added, I guess I didn't realize what you were doing there #2020-01-31 14:08:50 vlaaad it's main value over powershell-based runner is that it can run on default windows command line. We explored moving Defold to tools-deps from lein, but other tooling depends on cmd and can't run in pwsh #2020-01-31 14:09:16 borkdude the benefit of deps.exe (or deps.clj as source file) is that you don't have to maintain bash and/or PS #2020-01-31 14:09:34 alexmiller so in the graal case, what exactly is compiled there? #2020-01-31 14:09:35 borkdude but otherwise it's exactly the same thing behaviorally #2020-01-31 14:10:07 alexmiller is it launching a jvm to do the classpath construction like clj? #2020-01-31 14:12:10 alexmiller well, that's cool, I didn't realize all that #2020-01-31 14:12:26 alexmiller yeah #2020-01-31 14:12:51 alexmiller well, I'll put it in the thinker then #2020-01-31 14:13:11 alexmiller need to spend some time working on clojure instead! #2020-01-31 14:19:46 alexmiller the potentially interesting variant here (much bigger lift) is to combine the deps.clj stuff with the classpath construction and graal all of that (but I'm sure there are a lot more challenges to do that) #2020-01-31 14:30:02 vlaaad knowing @borkdude I would expect a release announcement tomorrow 😆 #2020-01-31 14:31:10 borkdude yeah, I tried to graal the tools jar as well, but got into some weird stuff which I didn't explore in much detail #2020-01-31 14:33:34 borkdude I think it was related to the jgit library, but I only remember vaguely #2020-02-01 21:12:42 lvh Is there a way to express Maven plugins in deps.edn so that -Spom will include the plugin (and a repo) or do I want to do that by editing the resulting XML #2020-02-01 21:17:29 borkdude Another way would be to use clojure.data.xml in a script to manipulate the generated pom afterwards #2020-02-01 21:17:58 lvh yep that's what I had in mind with "editing" #2020-02-01 21:18:09 lvh I take it clojure.data.xml works great with babashka/sci? 🙂 #2020-02-01 21:21:22 borkdude I'd just use a normal clojure script for it right now. #2020-02-01 21:21:38 lvh right, easy enough to port later #2020-02-01 22:23:33 alexmiller -Spom will sync into an existing pom, so just put what you want in the pom #2020-02-01 22:23:44 alexmiller It should stay there #2020-02-02 12:39:53 borkdude to use the script with clojure (JVM) instead of bb, you'll need to change the shebang into #!/usr/bin/env clojure, add a prn around the returning expression and add (shutdown-agents) at the bottom #2020-02-04 10:47:10 holyjak How do you folks search for tools that build on tools deps, such as clj-new and depstar? And/or is there a list of these tools somewhere? #2020-02-04 10:50:09 borkdude @holyjak this is documented on the wiki #2020-02-04 11:24:06 holyjak Thank you. I have sent a PR to the Deps reference to make it easier to find the page :-) #2020-02-04 15:28:23 Derek Passen What’s the recommended way of mixing in a dev resources directory? I have a :dev alias that has :extra-paths ["dev"] but I can’t call ( "local.edn") assuming that file lives within ./dev/resources. #2020-02-04 15:31:40 alexmiller include it in :extra-paths in an alias #2020-02-04 15:31:54 Derek Passen “dev/resources” ? #2020-02-04 15:32:06 alexmiller well, whatever the path is #2020-02-04 15:32:16 Derek Passen I already have “dev” as an extra-path #2020-02-04 15:32:19 alexmiller sorry, re-reading, yes #2020-02-04 15:32:36 Derek Passen thanks alex, I will give that a try #2020-02-04 15:32:42 alexmiller the paths are classpath roots, so any file under that path has to be relative to that root #2020-02-04 15:33:05 alexmiller so either dev and resources/local.edn or dev/resources and local.edn #2020-02-04 15:33:13 Derek Passen makes sense #2020-02-07 09:47:51 robert-stuttaford how do folks document the reason for a dep in their deps.edn? i was thinking we could just add a :doc "For feature x" key to the map alongside :mvn/version et al #2020-02-07 09:50:42 vlaaad ;; need this lib because x #2020-02-07 10:14:57 danieroux I have this:
org.clojure/tools.reader {:mvn/version "1.0.0-beta4" :dictated-by :datomic} #2020-02-07 10:19:59 robert-stuttaford i like that! what do you do for deps that are for specific feature code, rather than to support another dep? #2020-02-07 10:26:47 dominicm I don't! #2020-02-07 10:27:29 robert-stuttaford separate question: is there a dependency conflict diagnosis output in the tools.deps world, like the one lein has when you lein deps :tree? #2020-02-07 10:27:42 dominicm I think there's a ticket for this #2020-02-07 10:29:31 robert-stuttaford ok cool - this is probably the very last thing we'll still need lein for! #2020-02-07 10:30:32 dominicm Lein handles conflicts very differently #2020-02-07 10:30:39 dominicm I wouldn't compare the outputs at all #2020-02-07 10:31:06 delaguardo there is a way to generate tracefile with https://github.com/clojure/tools.deps.graph
right now there is no tooling to work with those traces and you have to analyze it yourself but good to know as an alternative #2020-02-07 10:31:40 dominicm Personally, I've stopped having problems when using deps.edn. It has better conflict resolution behaviour. #2020-02-07 10:32:02 dominicm Yeah ^^ trace file has the data. #2020-02-07 10:32:50 robert-stuttaford fantastic #2020-02-07 10:27:29 dominicm You're creating 2 things you need to sync up. You won't remember when you need a new feature. #2020-02-07 10:28:23 robert-stuttaford i agree, @dominicm -- the counter argument though is when upgrading dependencies, it's a lot of effort to dig around and understand how they are used. so i'm not planning reams of content. just enough to understand its importance and connection #2020-02-07 10:29:53 dominicm On my list is the job to solve that through tooling :) #2020-02-07 10:30:13 dominicm Kondo should make it fairly trivial to do at the namespace level #2020-02-07 10:30:26 robert-stuttaford great point! #2020-02-07 10:32:28 robert-stuttaford you're like Varys from Game of Thrones, with little birds telling you when people talk about kondo 😄 #2020-02-07 10:32:51 borkdude I will now crawl back under my rock #2020-02-07 10:33:04 robert-stuttaford hahah i think it's great! #2020-02-07 10:32:49 dominicm The trick is figuring out which jars have which namespaces defined within, then mapping that to the dependency #2020-02-07 10:33:12 dominicm Kondo supports jars though, so it isn't too difficult. #2020-02-07 10:34:26 borkdude kondo also reports in which files the namepace are defined. it also lists the jar when you use the option {:output {:canonical-paths true}} #2020-02-07 22:25:18 dominicm Might be better to not use kondo actually, as then I'll miss datomic and other AOT libraries #2020-02-07 10:39:38 borkdude the jar string is a little bit weird though, I want to make it more compatible with https://java.net, so it might change a little #2020-02-07 16:30:20 dominicm It should probably be the same as what io/resource would give me #2020-02-07 17:06:57 borkdude yeah, that's what I want to make it #2020-02-07 17:07:05 borkdude so expect a small breaking change there #2020-02-07 17:09:01 dominicm Okay, won't write my code this weekend then. #2020-02-07 17:09:40 borkdude it's only a small change, you can still use it to compare if it came from a different jar, to e.g. discover namespace conflicts #2020-02-07 17:10:19 borkdude jar:foo:bar will become something like <jar://file>://foo#!/dude or something, I can't remember these details #2020-02-07 17:10:46 dominicm I do that with sed atm. I think you really want to compare jars for conflicts at the file level. #2020-02-07 17:11:18 borkdude maybe you can just specify what you need, so I can take that into account #2020-02-07 17:11:31 borkdude if something isn't working for that purpose right now #2020-02-07 17:16:52 dominicm I could make what's there work, don't worry. I'm just patient enough. http://Java.net compatibility would be good.
What I'm planning on doing is linking the jar back to its dependency (eg bidi/bidi), then I know which namespaces use which dependency. #2020-02-07 21:51:16 souenzzo I just realized that {} is a enough deps.edn for some libraries #2020-02-07 21:54:10 alexmiller that was the idea :) #2020-02-07 21:54:29 alexmiller if you have no deps but Clojure and typical src dir, you should be good :) #2020-02-07 21:56:40 borkdude yep, it's awesome. and even if a library (like medley before not too long ago) doesn't have one, you can declare a manifest type #2020-02-07 22:41:06 seancorfield @souenzzo or even no deps.edn at all 🙂
(! 1222)-> mkdir -p example/src/simple
(! 1223)-> echo '(ns simple.main) (defn -main [& args] (println "Hello, World!"))' > example/src/simple/main.clj
(! 1224)-> cd example/
(! 1225)-> ls
src
(! 1226)-> clojure -m simple.main
Hello, World!
#2020-02-07 22:42:23 souenzzo But without a file with a empty map you cant import as a git-dep #2020-02-07 22:43:21 seancorfield Yes you can. You can specify a manifest type of :deps #2020-02-07 22:43:43 borkdude You can:
Local project coordinate: {:local/root "/path/to/project"}
Optional key :deps/manifest
Specifies the project manifest type
Default is to auto-detect the project type (currently either :deps or :pom) #2020-02-07 22:44:08 seancorfield Dang, you're fast! I was just about to paste that in! 🙂 #2020-02-07 22:44:49 borkdude It was still in my clipboard from the previous time someone asked (kidding) #2020-02-07 22:47:11 seancorfield For example
(! 1234)-> clojure -Sdeps '{:deps {jedi-time {:git/url "" :sha "e3c9ab41267be7d9033a13e1d666a3411f53d3c6"}}}'
Cloning:
...
Error building classpath. Manifest type not detected when finding deps for jedi-time/jedi-time in coordinate {:git/url "", :sha "e3c9ab41267be7d9033a13e1d666a3411f53d3c6"}
(! 1235)-> clojure -Sdeps '{:deps {jedi-time {:git/url "" :sha "e3c9ab41267be7d9033a13e1d666a3411f53d3c6" :deps/manifest :deps}}}'
Clojure 1.10.1
user=> #2020-02-07 22:45:41 borkdude Would it make sense for tools.deps to default manifest type to deps.edn if a project has no pom.xml / deps.edn? Why not? #2020-02-07 22:46:47 ghadi if there's no manifest, why default the manifest type? #2020-02-07 22:47:17 ghadi there's an implication/assumption I don't get #2020-02-07 22:48:00 borkdude in the case of git deps with only a src folder, that saves you one file or some config 😛 probably a stupid reason #2020-02-07 22:48:19 ghadi sounds stupid (you said it 🙂 ) #2020-02-07 22:48:19 seancorfield I think defaulting to :deps would lead to confusing errors when someone tries to mistakenly depend on a Leiningen project that had other dependencies. #2020-02-07 22:49:04 ghadi getting a magic ambient default seems like an undesired thing... #2020-02-07 22:50:26 souenzzo there is only one default:
paths ["src"] #2020-02-07 22:50:27 borkdude the reason I brought it up is that this option is often not known to people, maybe because the guide isn't very elaborate about it, although it does document it #2020-02-07 22:52:34 andy.fingerhut If http://Clojure.org folks are open to a PR enhancing the docs, and you are open to writing such an enhancement, I see magic occurring in the near future 🙂 #2020-02-07 22:53:28 ghadi I guess I don’t understand the problem enough to evaluate a solution #2020-02-07 22:53:49 ghadi If you control the dir you can add a manifest easily #2020-02-07 22:55:30 borkdude I was thinking the other way around: sometimes it's desirable to depend on a clojure project out of your control that only has a project.clj but no pom and no dependencies.
what I did for medley is add a deps.edn to it: https://github.com/weavejester/medley/commits/master
because that was more convenient to me than having to look up the exact option for manifest type again 😃 #2020-02-07 22:56:00 borkdude anyway, not that important I guess. have a good weekend y'all #2020-02-07 22:56:32 ghadi It’s been a while since I tried but I though project.clj’s were read automatically #2020-02-07 22:56:39 ghadi See ya @borkdude #2020-02-07 22:57:06 andy.fingerhut I don't think project.clj files are read if you depend upon a git commit SHA of a Leiningen library that has no deps.edn, is it? #2020-02-07 22:57:49 andy.fingerhut maybe tools.deps has a way to do that I do not realize, but it seems like that would be kind of magical ambient default, wouldn't it? #2020-02-07 23:00:19 andy.fingerhut I could be way off here, but my guess was that a Leiningen library deployed as a JAR, e.g. to http://clojars.org, typically contains a pom.xml that tools.deps would use. #2020-02-07 23:00:23 alexmiller project.clj's are not read now #2020-02-07 23:00:31 alexmiller but maybe they will be in the future #2020-02-07 23:01:15 alexmiller I don't really want to change the defaults until that's sorted #2020-02-07 23:01:37 andy.fingerhut I'm sure you know that project.clj files can contain arbitrary executable code... #2020-02-07 23:03:23 borkdude so do sources files 😉 #2020-02-07 23:03:34 andy.fingerhut deps.edn files don't, though. #2020-02-07 23:03:43 andy.fingerhut nor do pom.xml files #2020-02-07 23:05:42 alexmiller less true than you think #2020-02-07 23:06:04 alexmiller pom model resolution is remarkably f'ing complicated #2020-02-07 23:06:14 alexmiller and the libs around it, more so #2020-02-07 23:07:05 andy.fingerhut OK, news to me, but I am pretty sure you have looked at it 1000x longer than I have 🙂 I was assuming that pom.xml didn't have executable / evaluatable pieces to them. #2020-02-07 23:08:59 alexmiller to actually resolve pom files the way mvn does, you need to talk to the web, download plugins + parent poms, etc #2020-02-07 23:09:16 alexmiller and there are a lot of properties that need to be interpolated as well #2020-02-07 23:10:54 alexmiller tools.deps is tapping into apis to do the right thing for most of that (and most people don't do anything very fancy) #2020-02-07 23:04:24 borkdude maybe more people should get into the habit of checking in the generated pom.xml files if they want their non deps.edn libs to be used #2020-02-07 23:05:22 andy.fingerhut FYI, I am not here asking for new tools.deps features -- just checking whether I am missing something in what it does now. #2020-02-07 23:06:43 alexmiller I actually think it would be good to have project.clj support #2020-02-07 23:07:28 alexmiller I am undecided whether it should be naive support (data only) that relies on no leiningen stuff or good (perhaps add-on) support that does the actually right thing #2020-02-07 23:07:34 alexmiller or both #2020-02-07 23:08:21 borkdude naive as in only read the :dependencies vector and be done with it? #2020-02-07 23:10:43 andy.fingerhut I would guesstimate that over 90% of project.clj files start with (defproject and do not rely on code execution for the value of their top level :dependencies key, but haven't done any kind of wide survey. #2020-02-07 23:09:43 borkdude would cover the first 80% probably #2020-02-07 23:11:18 alexmiller yeah. honestly I would love to know the actual number. #2020-02-07 23:11:58 borkdude sandboxed execution of project.clj could also work for the remaining 20% #2020-02-07 23:12:30 alexmiller you mean eval? #2020-02-07 23:14:04 borkdude yes. personally I would think it would be ok to only accept EDN-parsable project.cljs and ignore those with read-evals. #2020-02-07 23:14:21 andy.fingerhut I am sure developers often download source of a Leiningen project and run Lein commands before looking whether project.clj does code execution, but at least they had to download the source for that to happen. Doing eval on the project.clj of transitive dependencies is another level of openness to exploits. #2020-02-07 23:14:23 alexmiller there's a fairly easy path to do the right thing using the lein code base iirc, but it does require depending on that code (which pulls in a bunch of stuff) #2020-02-07 23:15:20 alexmiller doing this the really really right way (like eval in clojail or something) is probably not something I personally am ever going to write :) #2020-02-07 23:15:36 borkdude that's why I said "sandboxed", something like clojail or sci, so allow only a restricted set of functions. it sounds like a bit of a headache that may not be worth it for the 20% or less #2020-02-07 23:15:59 andy.fingerhut think of the endless bug reports ... #2020-02-07 23:15:59 borkdude same thoughts there #2020-02-07 23:16:05 alexmiller like I said, I'd be interested in knowing a better estimate of the % #2020-02-07 23:16:27 alexmiller if it's 99% then that's an easier decision #2020-02-07 23:16:34 borkdude maybe it's more like 5% or 1% #2020-02-07 23:16:42 andy.fingerhut Is the % number you are interested in mainly "is pure edn" vs. not? Or some finer distinctions would be interesting to know? #2020-02-07 23:17:22 borkdude I guess you can retrieve a list of all clojars projects and then curl to their github repo and try to parse it with edn/read-string #2020-02-07 23:18:28 borkdude I have swapped from read-eval to just templating an EDN-readable project.clj recently. the templating is done with clojure ironically 🙂 #2020-02-07 23:18:36 alexmiller just whether a naive slurp edn/read find-:dependencies would work #2020-02-07 23:19:46 alexmiller if anyone wanted to get a believable estimate and write that code, I'm happy to consider it #2020-02-07 23:19:49 andy.fingerhut and likely ignore any :profiles with their own :dependencies keys #2020-02-07 23:20:11 alexmiller not doing anything down that line #2020-02-07 23:21:09 alexmiller gotta run... later #2020-02-07 23:21:19 borkdude just so we're clear, this is about using project.clj projects as git deps right? #2020-02-07 23:21:40 borkdude that would probably exclude projects that have java sources in them #2020-02-07 23:21:53 andy.fingerhut @borkdude I am happy to work on my other hobbies if this has set a fire under you. You would probably beat me to the answer, anyway 🙂 One perhaps additional interesting thing to check, if it isn't difficult, is whether the http://clojars.org pom.xml is "equivalent" to the top level :dependencies key. #2020-02-07 23:22:40 borkdude @andy.fingerhut feel free to give this a go, I'm not doing this anytime soon, got lots of other stuff to do atm #2020-02-07 23:26:56 borkdude I didn't even ask for project.clj support btw, hehe #2020-02-07 23:27:09 borkdude Although I would not be against it #2020-02-07 23:28:48 andy.fingerhut That's the beauty of it -- neither of us did, but Alex was already wondering about doing it 🙂 #2020-02-07 23:29:56 borkdude 😃 #2020-02-07 23:23:15 andy.fingerhut As a sanity check on whether people often deploy pom.xml files that are a different dependencies list than the top level :dependencies key #2020-02-07 23:26:11 borkdude Usually when I need a project.clj as git dep really bad, I just fork it and add the deps.edn myself #2020-02-07 23:32:05 borkdude Not directly related to this problem, but there is a repo which collects stats from clojars as edn files:
https://github.com/plexus/clojars-stats
Could be of use for some kinds of analyses #2020-02-07 23:35:19 andy.fingerhut Thanks for pointing that out. If nothing else, it might be a good way to get a large list of projects, and sort them by download popularity or something like that. #2020-02-07 23:35:48 andy.fingerhut and drive a program that downloaded more stuff about each project found there. #2020-02-08 19:49:50 mss is there a way to specify a path to a specific deps.edn file when using clj? #2020-02-08 19:52:33 alexmiller no #2020-02-08 19:52:55 alexmiller what's the use case? #2020-02-08 20:27:36 mss writing a bash script in one dir to build a module at another path #2020-02-08 20:27:50 mss obv easy to work around, was just wondering if I was missing something obvious. thanks! #2020-02-08 20:41:29 Nate Sutton I have multiple single-file projects in the same directory each with their own deps file. I have to read the deps file in and then use the contents as an argument in order to work around not having a way to specify a deps file #2020-02-08 20:42:04 Nate Sutton so, that's a use case #2020-02-08 20:50:44 borkdude @mss fwiw, deps.clj does support this with the non-standard -Sdeps-file option #2020-02-08 21:13:15 seancorfield @mss If you do CLJ_CONFIG=/path/to/other/folder cllj it will look there for the user-level deps.edn (instead of you home directory) which might give you part of the solution? #2020-02-08 21:19:43 seancorfield I like the -Sdeps-file idea tho'. I would use that quite often I suspect if clojure supported it. @alexmiller would that even be considered as an enhancement? #2020-02-08 21:22:34 borkdude I often use it when I need the classpath from some other project in some other directory to lint with clj-kondo #2020-02-08 21:23:14 borkdude although $ ( cd /tmp/project ; clojure -Spath ) works too #2020-02-08 21:24:28 seancorfield I suppose you can always use clj -Sdeps "$(cat /path/to/deps.edn)" #2020-02-08 21:26:18 borkdude that works, but your local deps.edn will still be merged in, which is not always what I want #2020-02-08 21:26:37 borkdude -Sdeps-file will regard that file instead of the local deps.edn #2020-02-08 21:28:52 seancorfield Yeah, there are a lot of possibilities around which combination of files you might want to work from. #2020-02-08 21:55:32 alexmiller I've walked down the path of adding this a couple times for different reasons but there are a lot of unobvious consequences #2020-02-08 21:55:56 alexmiller like do you interpret relative paths in deps.edn from the deps.edn location or current location? #2020-02-08 21:56:41 alexmiller there are also now a lot of code locations that assume the deps.edn name, not just in tools.deps.alpha but in external tools around it #2020-02-08 21:58:33 alexmiller at the moment the tradeoffs don't seem worth it to me but I am interested to hear about cases where people need stuff like this though so I can continue to evaluate that #2020-02-09 00:58:31 Nate Sutton I think the relative path question could be solved by splitting the concepts of which deps file to use from the location of the project root #2020-02-09 00:59:15 Nate Sutton I agree it expands the scope, though #2020-02-09 01:02:20 Nate Sutton it's alpha, thankfully, so changes like this would be ok? #2020-02-09 01:23:00 andy.fingerhut I suspect Alex is in a mind set where even though it is alpha, adding things that he doesn't see value in supporting long term is like intentionally planning to pull the rug out later. #2020-02-09 03:22:25 alexmiller it's not alpha in my mind #2020-02-09 03:22:28 alexmiller ;) #2020-02-09 03:23:04 alexmiller Rich and I just haven't had the time to think through what changing it to be non-alpha means yet #2020-02-09 06:54:39 Nate Sutton it's not alpha?? #2020-02-09 07:51:02 andy.fingerhut If I interpret what Alex said correctly, it is currently alpha, but he and Rich want it to become non-alpha soon-ish. #2020-02-09 07:56:17 alexmiller it has alpha in the name, but I have been maintaining it like it's not alpha for a long time, and believe the api is stable #2020-02-09 08:08:38 borkdude deps.clj non-standard features are certainly alpha though, I think it can be a place of experimentation as far as the bash part goes #2020-02-09 10:07:19 peterwestmacott Morning! I've just got an error from my IDE saying: "The following errors were found during project resolve: /home/peter/personal/mandel-zero/deps.edn: Destination path "see" already exists and is not an empty directory" - it doesn't tell me where this non-empty directory is though! Is there a place where tools-deps caches dependencies from github that I can go and delete things from? #2020-02-09 10:14:25 dharrigan Have you tried to delete .cpcache? #2020-02-09 10:16:38 peterwestmacott I think its ~/.gitlibs #2020-02-09 10:28:23 peterwestmacott oh, dear, I seem to have broken it. it now just says 'Auth fail' instead #2020-02-09 14:59:08 alexmiller what happens if you are just at the command line? #2020-02-09 15:00:25 alexmiller these sound like ide specific issues, not tools.deps #2020-02-09 15:20:55 peterwestmacott strangely not. The ide appears to be reporting an underlying error #2020-02-09 15:21:16 peterwestmacott on the plus side I do get a stack trace though #2020-02-09 15:21:25 peterwestmacott Cloning:
#2020-02-09 15:24:23 peterwestmacott tbf I did go deleting directories in ~/.gitlibs to try and fix an earlier problem, which was probably a little reckless - I was sort of expecting them to get recreated as needed #2020-02-09 15:48:32 alexmiller what git url are you using? #2020-02-09 15:52:44 alexmiller what os are you on? #2020-02-09 15:54:15 alexmiller is your ssh-agent running? does it have your ssh identity? #2020-02-09 15:54:26 alexmiller ssh-add -l should list your key #2020-02-09 16:26:23 peterwestmacott The agent has no identities. #2020-02-09 16:26:30 peterwestmacott I'm guessing that's bad #2020-02-09 16:26:38 peterwestmacott I'm running xubuntu #2020-02-09 16:28:57 peterwestmacott oh, that seemed to sort it out. THANKS ALEX!!!! #2020-02-10 10:11:08 andy.fingerhut Is there a way in one's $HOME/.clojure/deps.edn file to have a :local/root dependency with a relative path name starting from your home directory? I tried {:local/root "$HOME/path/to/my.jar"} and {:local/root "~/path/to/my.jar"} , but neither seemed to work #2020-02-10 10:11:54 andy.fingerhut I ask because it would be helpful to have such a thing for sharing a ~/.clojure/deps.edn file contents across multiple systems, where my home directory absolute path name differs between them. #2020-02-10 10:14:50 borkdude tildes are not expanded in .edn files #2020-02-10 10:15:01 borkdude neither is $HOME #2020-02-10 10:16:03 borkdude but I get the problem, would be nice to see a solution for #2020-02-10 10:56:21 sogaiu my current work-around is to put all source under ~/src/ (or some other directory right under my home directory) and make paths relative: e.g. "../REBL/REBL.jar"
ofc, that means on each system source code needs to live under a directory right under ~.
on a related note, to get the effect of "variables" in deps.edn, i've been thinking whether to generate deps.edn (possibly from some template). #2020-02-10 10:57:29 borkdude with some lightweight scripting / templating you can get around it though #2020-02-10 10:58:32 borkdude ah, you also suggested that sogaiu 😉 #2020-02-10 13:45:21 alexmiller I'm going to continue to resist variables in deps.edn for as long as possible :) #2020-02-10 17:52:06 andy.fingerhut Is tilde as alias for home directory in same category? #2020-02-10 17:52:24 andy.fingerhut Not an urgent need by any means, just asking while we are in the neighborhood. #2020-02-10 17:52:24 alexmiller yes #2020-02-10 17:52:47 alexmiller it's in the category of "processing step beyond reading as data" #2020-02-10 17:53:42 alexmiller having none of those is a huge advantage for downstream work #2020-02-10 20:47:04 mike_ananev Just hacking my own build tool right now, and I created new file build.edn and code for processing with comment in code
"This sections dropped to build.edn cause Alex Miller continue to resist variables in deps.edn"
😎 #2020-02-10 20:50:55 andy.fingerhut I mean, you can give that as the reason if you want, but as an engineer, I'd say addressing the reasons for that design decision, rather than focusing on the person that made them, is a good practice. Sure, include the name of one or more people responsible for making the decisions if you want, but without the reasons, it can sound like you believe the decisions are arbitrary, or not grounded in any kinds of reasons. Not saying you have done that -- just pointing out the possibilities of misinterpretation. #2020-02-10 14:03:03 dominicm I think generating your own deps.edn is a good idea #2020-02-10 16:08:38 sparkofreason Are there tools/methodologies around adapting lein projects containing java to be amenable to clj/`deps.edn` ? I'm looking at a potential dependency that is fairly new and fast-moving, would like to be able to reliably pin my reference to a known SHA, so I can reference the code at that point rather than depending on potentially out-of-date docs, etc. I could manually run lein and deploy it locally, but something more automated where I can just update the SHA and have clj do the right thing would be nice-to-have. #2020-02-10 16:15:30 borkdude @dave.dixon The most reliable way I know is fork it and add a deps.edn to it. #2020-02-10 16:17:40 sparkofreason Yes. But then how do I get the downstream project to compile the java and put the classes on the classpath? I think I could do it for the specific case of that project using an alias and a tool like badigeon, but curious if anybody had come up with anything more general. #2020-02-10 16:18:43 borkdude if it contains java, it's a bit of a different story. I would just deploy it as a leiningen artifact to your own org on clojars in this case #2020-02-10 16:19:20 borkdude (my personal take on it, for compiling java I always use lein, since it just works) #2020-02-10 18:32:42 sparkofreason Sort of hacked it by building the java classes, and adding that directory to the :paths . Pushed to github, and that works. Curious if there's any reason to not do this. #2020-02-11 09:44:36 rickmoynihan > Curious if there’s any reason to not do this.
It’s not good to commit complied (and binary) artefacts into source control.
It would be better to compile the java classes and deploy it as a library. Either a shell script with javac and some mvn calls, or just move the java code into its own lein project and use lein & project.clj to compile and deploy the java bit, and depend on it from the tools.deps lib.
Whether to break out the java as a separate lib largely depends on how stable it is, and how often you expect to change it etc. #2020-02-11 14:36:10 sparkofreason > It’s not good to commit complied (and binary) artefacts into source control.
I've heard this, and my understanding (maybe wrong) for the rationale is that git doesn't handle large binaries well, as might be the case for a jar or fully linked executable. Class files, however, are pretty small, and map closely to the corresponding java source. #2020-02-11 14:43:03 sparkofreason The key risk is having the compiled class files be out of sync with the source. But that's kind of the problem I'm facing anyway, source is changing, and the clojars artifacts may not match, which makes it hard to figure out which features are there etc. More ideal would be a tool that built the java classes after clj pulled from the indicated :sha. #2020-02-11 14:56:41 rickmoynihan Well there are a whole bunch of reasons:
1. git doesn’t handle large files in a repo very well (operations on git repos slow to a crawl on large repo’s, e.g. changing branches etc)
2. you can’t diff a binary very well, so merging branches etc becomes painful. Should you accept a change or not?
3. you have a dependency between source and output files in the repo
4. related to 3, but for non-binary output files; e.g. compiling and minifying css/js you get huge diffs in your repo that can cause some tools to crash/struggle to render. You can set attributes on the files in git, but not all tools honor them properly.
5. different versions of javac generate different byte code for the same source code so you may get diffs and deltas just by upgrading java versions
6. tools.deps can’t handle files in a repo over ~100mb (due to limitations in jgit)
7. adopting this practice on repo early on, normalises the behaviour, and all sorts of binaries get commited; git performance tanks, and then you have to rewrite the whole history with filter-branch, or worse because of lack of patience around rewriting the history correctly the team decide to drop all the git history and start fresh in a new repo.
I’ve worked in many repo’s where this has been done, and it’s almost always caused problems. Not saying you won’t get away it; just be aware of the problems. #2020-02-11 14:57:44 rickmoynihan I’ve also personally experienced all of those at some point in the last 15+ years or so using git. #2020-02-11 16:29:21 sparkofreason Agreed in general. And most of this sounds like something that should be fixed in git, making the abstraction less leaky, but that's a whole other thread. Clearly it would be better to have a way to trigger the java build after clj pulls and update the classpath accordingly. #2020-02-11 16:39:48 rickmoynihan There are fixes in/around git for things like large-files, and git handles binaries ok… However you can’t really solve the issue of how to merge a random binary; it’s essentially pick one or the other or commit a new one. I’m not sure git can really do much more to help there.
For me philosophically it’s just the wrong thing to do; git is for source control and generated files aren’t source. I’m saying this as someone who has committed binaries into repos though; and may at some point do it again… I’d just always very much prefer to avoid it. #2020-02-10 17:56:15 andy.fingerhut I have not been working on this full time by any means, but from a programmatically generated file from http://Clojars.org, I have a list of 18,493 projects and their URLs they list in the artifact there, which appear to be on Github, downloading now. This is partial progress towards answering the question "what fraction of Leiningen project.clj files are readable using clojure.edn/read". #2020-02-10 17:57:58 andy.fingerhut As an aside on the kinds of weird things I now expect more often in such large-ish data sets: 672 projects with the incorrect URL #2020-02-10 18:23:15 andy.fingerhut 447 have this as the only URL my code finds in the data about the artifact, from Leiningen's template, it appears: #2020-02-10 18:24:22 andy.fingerhut Those are all outside of my list of 18,493, which is reduced from the original list size of 25,675. 18K seemed like a representative subset 🙂 #2020-02-10 17:57:34 dominicm It might be quicker to just hit up the github url for project.clj #2020-02-10 17:58:06 andy.fingerhut Not at this point it wouldn't be 🙂 #2020-02-10 17:58:50 andy.fingerhut And I am curious to ask more questions than about just the project.clj file, too, e.g. how many have a deps.edn or pom.xml file checked into the source repo? #2020-02-10 17:59:26 andy.fingerhut Basically I am not going for quick results here, but the potential for thorough follow up question asking capability. #2020-02-10 18:01:48 andy.fingerhut ... but I should first say: thanks for the suggestion. I had not considered that, and might be useful later. #2020-02-10 18:03:47 dominicm The idea of cloning all those repos hurts my disk #2020-02-10 18:07:39 andy.fingerhut Somewhere on a back disk or three of mine, I have about half a terabyte of Internet packet traces recorded at various times and places from live links on the Internet, headers only, plus timestamps, used in some research on various things in computer networking. Disk space is pretty cheap 🙂 #2020-02-10 20:43:16 sam I’m trying to run a project with a private git dependency on Windows (in Powershell). The doc says “ssh authentication works by connecting to the local ssh agent (ssh-agent on nix or *Pageant via PuTTY on Windows)”
I tried creating a key pair in puttyGen, added it to Pageant, (and of course added the public key to my github account).
When that didn’t work, I tried generating a new key pair with ssh-keygen and adding it to the openssh agent via ssh-add.
I think the relevant part of the stacktrace is:
Caused by: org.eclipse.jgit.errors.TransportException: : Authentication is required but no CredentialsProvider has been registered
#2020-02-10 20:44:07 sam Is there something I’m missing? I was able to get it working fine on my mac. #2020-02-10 20:44:51 dominicm That's https, you need to use ssh @samwagg0583 #2020-02-10 20:45:01 dominicm Change the url to be the ssh one on github #2020-02-10 20:45:40 sam oh wow, I can’t believer I did that and have been spinning my wheels for hours! #2020-02-10 20:51:40 sam That did it, @dominicm. I’m embarrased, but thanks so much. #2020-02-10 22:12:12 hlolli one nix package (Lumo) suddenly started to fail with
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.clojure:clojure:pom:1.10.1 from/to central (): : Name or service not known
I'm using tools-deps to resolve the dependencies, but I highly doubt this is related to tools-deps. But could be some mirror setting I guess? This at least isn't network error since Hydra (the CI of nixpkgs) also reported this exact same error. Maybe this error sounds familiar to someone?
Full stack here 4wiw https://pastebin.com/sC6d0Cdn #2020-02-10 22:15:23 andy.fingerhut In a web browser, I can go to
successfully. Maybe some transient failure, or networking problem between that host and public Internet? #2020-02-10 22:17:39 ghadi you need to paste your settings for fetching these deps #2020-02-10 22:18:01 ghadi or whatever nixpkgs is doing #2020-02-10 22:18:48 ghadi seems pretty clear:
Caused by: https://java.net.UnknownHostException: https://repo1.maven.org: Name or service not known
at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) #2020-02-10 22:19:32 ghadi either transient name resolution failure, or JVM is mistakenly resolving maven through IPv6 instead of IPv4 #2020-02-10 22:19:43 hlolli let repos = [
""
""
""
""
""
];
....
{
name = "org.clojure/clojure";
path = pkgs.fetchMavenArtifact {
inherit repos;
artifactId = "clojure";
groupId = "org.clojure";
sha512 = "f28178179483531862afae13e246386f8fda081afa523d3c4ea3a083ab607d23575d38ecb9ec0ee7f4d65cbe39a119f680e6de4669bc9cf593aa92be0c61562b";
version = "1.10.1";
};
}
...
so I download the jars then call clojure basically #2020-02-10 22:20:46 ghadi also the error you're pasting is a different version than the one in nix #2020-02-10 22:21:01 ghadi 1.10.1 vs 1.10.0-beta5 #2020-02-10 22:21:06 hlolli the latter one uses beta, yes, same error still. I tried bumping. #2020-02-10 22:21:11 ghadi but still, the error cause is accurate #2020-02-10 22:21:17 ghadi it has nothing to do with the version #2020-02-10 22:21:25 ghadi there's a name resolution error #2020-02-10 22:21:47 hlolli yes, something in that direction was my instinct #2020-02-10 22:22:05 hlolli just dns lookup? #2020-02-10 22:22:16 ghadi instinct / gut feeling unnecessary 🙂
you just have to read the stacktrace #2020-02-10 22:22:27 ghadi it appears like it's resolving through IPv6 #2020-02-10 22:22:40 andy.fingerhut Well, or it is using IPv6 when IPv6 is disabled somewhere in the network between you and DNS resolver, or between your host and where JAR file is. #2020-02-10 22:22:40 ghadi not sure if the JVM always appears like it's resolving through IPv6 #2020-02-10 22:22:51 hlolli well, I was only explaining why the first paste was in diferent version, but yeh, nevermind #2020-02-10 22:23:27 ghadi you can try to set -.preferIPv4Stack=true on the JVM to see if it resolves differently #2020-02-10 22:23:54 ghadi or try to resolve something that you know is invalid, and see if you get the same trace #2020-02-10 22:24:21 hlolli nice, that sounds like something I can try. I'm using 8.8.8.8 like always. Maybe maven isn't discoverable there (for me dns lookups are black boxes) #2020-02-10 22:24:38 hlolli thanks for the tip fellas 🙂 #2020-02-10 22:25:01 ghadi np. you should test your hypothesis #2020-02-10 22:25:34 ghadi also check if nix changed their global ipv6 settings or something #2020-02-10 22:26:12 hlolli that's also plausible, tough it could only be the package manager that could have such power, all else is configured by the user #2020-02-10 22:27:29 hlolli it's a beginning of a bug hunt I guess, again, thanks #2020-02-10 22:28:57 andy.fingerhut Sounds like it. We can throw the hot potato into what looks like the most likely hands for the problem, which at least in this case strongly appears not to be deps.edn code, nor any other particular package manager type of thing. #2020-02-10 22:31:19 alexmiller I was on a thread last week about some Maven deploy errors (different problem) and it sounded like they've been doing some server updates. could be something on their side. They have a jira system for reporting issues if you wanted to ask there. https://issues.sonatype.org/secure/Dashboard.jspa #2020-02-10 22:34:08 ghadi as a first step you should try to figure out what DNS settings you have in /etc/resolve* and what address types you have set with ip addr show #2020-02-10 23:04:14 hlolli reddest herring this week, I actually had stable and good connection to maven, the error was arriving at installation time and not build time, which in nix mean no unwanted side-effect (calling http(s)). So to take my quesiton to a new direction, why would the clojure executeable try to download clojure-1.10.1 when I specify its jar location on the classpath?
My classpath is (stored as classp)
/nix/store/58c2jsc0gq4mi73ma6pvphgnvnpwlnn8-com_cognitect_transit-java-0.8.332.jar:/nix/store/bayaxp12xw9kpbxn35pwab2s4ya8gpcn-org_clojure_data_json-0.2.6.jar:/nix/store/z26fq48r5xs7syp1j2df5pjv1qk37pjf-org_clojure_clojure-1.10.1.jar
...elided...
and command is
${clojure}/bin/clojure -Scp ${classp} -e "${compileClojurescript true}" #2020-02-10 23:08:49 hlolli well, I'll open a ticket if I find a clear regression between previous versions. No need to solve it here now. #2020-02-10 23:08:54 andy.fingerhut As a guess (I do not know implementation details), but there might be cases where clojure finds a different version in local file system, and time stamps or I-don't-yet-know-what cause it to make network connections to see if a newer version isa available? #2020-02-10 23:09:46 hlolli that could be, for security resons, all file timstamps are set to 0 (1jan1970). #2020-02-10 23:10:02 andy.fingerhut If you want to absolutely prevent this possibility, you could use clojure command to create a classpath as its output, and save that in a one-line bash script #2020-02-10 23:10:22 andy.fingerhut Using -Spath option. #2020-02-10 23:11:00 andy.fingerhut before you reach installation time, or whatever step doesn't like you to make network connections. #2020-02-10 23:11:16 hlolli hmm, good point #2020-02-10 23:12:31 hlolli that should be easy thing to test. The maven endpoints are initially made this way. I make tools-deps create classpath and maven endpoints which I have nix download for me, and then provide it again to clojure. Has worked fine last 2 years. #2020-02-10 23:13:45 hlolli then the question is if the generator and the build environment are mismatching, tough I updated 2 months ago, there's a chance that nixpkgs has even newer clojure data. #2020-02-10 23:13:53 andy.fingerhut Something changed, but I don't know what. Depending upon how much time you like tracking down such root causes, and how important it is, you can try, or just change steps to avoid it happening. #2020-02-11 00:03:50 alexmiller looking at the clojure script, I think it will still do staleness checking (whether one of the deps.edn files is newer than the cached classpath file), and potentially download and create an unused classpath file even if you -Scp #2020-02-11 00:06:02 alexmiller that seems like a bug, I don't think there's any reason to do that #2020-02-11 00:14:34 andy.fingerhut I was thinking one could use clojure -Spath ... > mypath.txt when you had network connections allowed, and "java -cp java command where you were not supposed to make network connections, but sure, making network connections unnecessarily in clojure / clj programs seems like a bad idea. #2020-02-11 00:19:06 alexmiller using clojure -Scp for the latter should have that effect #2020-02-11 05:21:34 richiardiandrea Maybe it is completely unrelated, but here we discovered that our Java mvn builds were throwing warnings cause we were still using #2020-02-11 05:26:58 andy.fingerhut Is https://repo1.maven.org supported still, and they are deprecating/phasing-out http? #2020-02-11 05:28:54 alexmiller http is long dead #2020-02-11 05:29:32 alexmiller they used to transfer you, but now it's just dead - that changed at end of 2019 I think #2020-02-11 06:07:44 richiardiandrea Yeah just http is dead 😁 #2020-02-11 04:22:16 cartesian-theatrics Hello,
I've been using the add-lib branch. Seems to be somewhat robust, but ran into an issue installing libpython-clj, which I figured I'd note:
user> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user> (add-lib 'clj-python/libpython-clj {:mvn/version "1.36"})
user> (require '[libpython-clj.require :refer [require-python]])
...
Error: failed to access class com.sun.jna.CallbackReference #2020-02-11 04:24:33 cartesian-theatrics Worked fine after restarting the repl with the dep. #2020-02-11 04:24:53 cartesian-theatrics Are there plans to ever merge add-lib into master? #2020-02-11 04:28:26 alexmiller it will probably be added in some form, but may change in api/impl #2020-02-11 04:30:10 cartesian-theatrics 👍 #2020-02-11 04:30:39 cartesian-theatrics It's still super useful in it's current form, so thanks for the tool! #2020-02-11 04:30:51 alexmiller add-lib will not always be able to safely add libs as it's trying to resolve new libs in terms of the already loaded library set #2020-02-11 04:33:09 alexmiller I'm not sure how the native stuff interacts with any of that either #2020-02-11 04:34:36 cartesian-theatrics I see. I'm not well versed on how deps are resolved, but I can imagine there's all sorts of edge cases. Thankfully it does very often work. #2020-02-11 04:39:00 seancorfield Agree. I do all my dev work with Atom/Chlorine + REBL and the add-lib branch of t.d.a. and I love being able to add new dependencies on the fly! Every now and then I add one and get false back (indicating it didn't add it for some reason) but mostly it works exactly as expected. #2020-02-11 04:40:40 seancorfield It would be nice if, instead of just false, add-lib returned more information about why it didn't add a dependency. #2020-02-11 04:41:30 cartesian-theatrics For sure, at some point think it would be great to give it (the feature) some signal boosting, especially if we can squash more of the edge-case issues. It really helps with flow. #2020-02-11 04:41:33 alexmiller well, it doesn't actually know why #2020-02-11 04:43:14 alexmiller it takes all currently loaded libs + the new lib and then does a full resolution, then compares the result with the prior set #2020-02-11 04:43:51 cartesian-theatrics Is there a conceivable "brute force" / descructive approach? #2020-02-11 04:44:43 seancorfield Fair enough @alexmiller -- it can't report what it does know! 🙂 #2020-02-11 04:45:01 alexmiller there are multiple possible reasons why you wouldn't end up with something new #2020-02-11 04:46:16 alexmiller because everything already loaded is "pinned", adding a different version of something already loaded isn't possible (starting from a real initial dep set doesn't have those constraints, so you can use the version selection to resolve) #2020-02-11 04:49:19 cartesian-theatrics I see. Sounds like the current form is about as good as we can do then. #2020-02-11 04:50:29 seancorfield @alexmiller Here's a repeatable case of failure, from an empty REPL with the add-lib branch:
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'jedi-time {:git/url "" :sha "8e756ecb71bbfa0b081e00d71a21c47037f1eae4" :deps/manifest :deps})
Cloning:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Checking out: at 8e756ecb71bbfa0b081e00d71a21c47037f1eae4
false #2020-02-11 04:51:58 seancorfield This works fine as a regular dep:
$ clj -Sdeps '{:deps {jedi-time {:git/url "" :sha "8e756ecb71bbfa0b081e00d71a21c47037f1eae4" :deps/manifest :deps}}}'
Clojure 1.10.1
user=> (require '[jedi-time.core :as jdt])
nil #2020-02-11 04:52:20 seancorfield Is it due to needing the :deps/manifest override? #2020-02-11 04:55:05 seancorfield (I see the add-lib branch is about two months behind master at this point) #2020-02-11 05:09:34 seancorfield (`add-lib` works if I use {:mvn/version "0.1.4"} instead of the git coordinates) #2020-02-11 05:15:29 andy.fingerhut OK, 13K GitHub repositories cloned, with URLs from Clojars projects, of which 12K have project.clj files in their home directories, ~500 deps.edn files in their home directories. More stats later. #2020-02-11 05:16:20 seancorfield @andy.fingerhut And the other ~500 are... boot? Or a mixture? #2020-02-11 05:18:03 andy.fingerhut Don't know yet. What file name should I look for? Is it boot.edn? #2020-02-11 05:18:05 seancorfield (are you counting deps.edn in projects that also have project.clj?) #2020-02-11 05:18:10 seancorfield build.boot #2020-02-11 05:18:18 andy.fingerhut No combinations checked for yet, but will do so. #2020-02-11 05:18:28 andy.fingerhut That was just raw count of those files in root dirs. #2020-02-11 05:18:28 seancorfield Sweet! You're a rock star! #2020-02-11 05:19:34 andy.fingerhut 635 build.boot files. Again no stats on projects with different combinations of those. As I said, bigger more detailed report I will send a link later, after I find out how many of these project.clj files can be read with clojure.edn/read #2020-02-11 05:20:08 andy.fingerhut And then I may want to get fancy and report not only the fraction, but also the weighted fraction, weighted by something like Clojars download counts. #2020-02-11 05:20:22 seancorfield Hmm, so CLI/`deps.edn` is already almost as popular as Boot. Interesting. #2020-02-11 05:21:32 andy.fingerhut And of course many of these repos may not have been updated for years... #2020-02-11 05:22:25 seancorfield Good point. Year-by-year stats based on last update would be interesting if you're thinking of going that way 🙂 #2020-02-11 05:24:46 andy.fingerhut That may be interesting enough to spend the extra time to get it. #2020-02-11 05:26:55 alexmiller deps is way more than boot in the 2020 survey #2020-02-11 05:27:23 seancorfield Good to know @alexmiller -- did you see my add-lib issue above (now scrolled off) #2020-02-11 05:32:22 alexmiller new deps-mgmt question was lein 84%, deps 49%, boot 6% #2020-02-11 05:35:24 seancorfield Wow! That's an impressive uptake! #2020-02-11 05:36:02 alexmiller and (obviously) that's multi-select, so presumably a lot of people using both lein and deps #2020-02-11 05:28:21 alexmiller I saw it, but not a priority for me right now - please file a question at https://ask.clojure.org #2020-02-11 18:50:11 alexmiller Thx #2020-02-11 05:29:08 seancorfield Gotcha. I'll see if I can narrow it down a bit first. Should git deps work, in general with add-lib? #2020-02-11 05:30:16 alexmiller off the top of my head, sure? #2020-02-11 05:33:19 alexmiller it's using the same resolve-deps as normal deps, so can't think why it wouldn't #2020-02-11 05:34:53 seancorfield Thanks. #2020-02-11 07:49:01 robert-stuttaford if i had an app project and a library project, and the library project had deps.edn aliases, could i invoke those from the app project? #2020-02-11 08:17:37 dominicm Nope. #2020-02-11 08:34:11 robert-stuttaford thanks for confirming! #2020-02-11 09:25:48 robert-stuttaford is there a canonical reference on how to use deps.edn git coordinates with private github repos? #2020-02-11 09:27:05 dominicm I think it's just as simple as using the ssh url #2020-02-11 09:38:46 robert-stuttaford thanks! #2020-02-11 13:19:08 souenzzo Hello
$ clj -Srepro -Sdeps '{:paths ["a" "b"] :deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}' -Spath | tr ':' '\n'
a
b
/home/souenzzo/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/home/souenzzo/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/a
/home/souenzzo/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/b
/home/souenzzo/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/home/souenzzo/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
a b paths from "root project" are "chained" to clj-kondo/lsp-server . I think that is a issue, maybe related to TDEPS-52
https://clojure.atlassian.net/browse/TDEPS-52 #2020-02-11 13:23:21 souenzzo I expect /home/souenzzo/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/src as the default :paths is ["src"]
May also be nice to control :deps/paths 👀 #2020-02-11 13:35:54 alexmiller what version of clj are you using? #2020-02-11 13:36:24 alexmiller clj -Sdescribe will tell you #2020-02-11 13:37:21 alexmiller if you're not using latest (1.10.1.507) could you update and try again? #2020-02-11 14:11:59 souenzzo I'm on 1.10.1.507
I posted a "onliner" so it's easy to rest in many versions #2020-02-11 13:38:05 alexmiller TDEPS-52 is long fixed, so I'm wondering if you're on an older version #2020-02-11 13:52:44 borkdude this is my output with the newest (507):
$ clj -Srepro -Sdeps '{:paths ["a" "b"] :deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}' -Spath | tr ':' '\n'
Cloning:
Checking out: at 4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5
a
b
/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/a
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/b
/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar #2020-02-11 13:53:27 borkdude so it might be an issue with :paths + :deps/root ? #2020-02-11 13:55:15 alexmiller dunno, must be a bug, please file either a jira or a question on https://ask.clojure.org #2020-02-11 13:55:41 alexmiller or I guess I'll just make a jira, nvm #2020-02-11 14:15:19 souenzzo atlassian login system is a pure mess
I have 3 accounts (clojure, work1, work2)
When I'm authed as ẃork1 and try to login to vote on clojure, it blows a auth error
then I need to go to work1 workspace, logout, login in clojure workspace, then(.................) 😓 #2020-02-11 14:22:59 souenzzo after create a new browser to login in jira, there is no vote option to vote on TDEPS issues? #2020-02-11 14:28:14 alexmiller the jira auth is for sure, not good, particularly cross accounts. identity is tied strongly to email but the gsuite tie-ins make everything pretty bad. I don't have a good solution to that other than private browser window #2020-02-11 14:28:59 alexmiller if you have a clojure jira logon, you should be able to vote. that said, they have been actively destroying the user experience with their "new ui" issue view. it could just be that it's buried a menu level or two down #2020-02-11 14:29:13 alexmiller I'm using the old ui, so not sure I can point to how to do it #2020-02-11 14:38:46 borkdude I have an old account, but I'm not able to vote for a long time anymore, at least, can't find that option #2020-02-11 14:44:45 souenzzo okay after spawn 5 firefox isolated instances and login with all my credentials, I'm sure that voting is disabled.
My account is {:tag :mailto:souenzzohotmail.comsouenzzohotmail.com, :attrs nil, :content nil} with "raw password" (without microsoft auth). #2020-02-11 15:17:31 alexmiller there is no way in jira to disable voting for a user #2020-02-11 15:17:47 alexmiller voting is enabled in our jira product instance #2020-02-11 15:18:28 alexmiller whether you can vote is dependent on whether you can edit issues and I have it configured in all projects that if you are a logged in user, you can edit issues #2020-02-11 15:18:46 alexmiller your acct looks normal - you have jira product access and good default roles like other users #2020-02-11 15:20:07 borkdude if voting is so hard to find, maybe we can just regard it as non-existing #2020-02-11 15:22:38 alexmiller it's a link at the top with the words "Vote for this issue" which seems pretty easy to find to me #2020-02-11 15:23:18 alexmiller (note that you can't vote for issues you create) #2020-02-11 15:23:44 borkdude I have tried to find it multiple times, and apparently I'm not the only one. I gave up on it. #2020-02-11 15:24:21 alexmiller well there must be something I'm missing then #2020-02-11 15:24:48 alexmiller there is no intent for it to not be there #2020-02-11 15:25:37 borkdude #2020-02-11 15:26:19 andy.fingerhut I see how to vote using my JIRA account when looking at a ticket using the old view, but not the new one #2020-02-11 15:27:03 andy.fingerhut There is a link "See the old view" near the top center of the frame. #2020-02-11 15:27:10 alexmiller yeah #2020-02-11 15:28:00 alexmiller turn OFF "New JIRA Issue view" - when you do, it asks you why. tell them you can't vote. :) #2020-02-11 15:28:19 andy.fingerhut But agreed without explicit instructions somewhere, most people won't bother hard trying to find it. #2020-02-11 15:31:24 alexmiller sorry, I turned this off so long ago I forgot about it #2020-02-11 15:31:58 alexmiller 278 votes to enable voting #2020-02-11 15:32:38 borkdude ok, I reverted to the old view of which I was unaware it existed and now I see the votes again #2020-02-11 15:32:49 andy.fingerhut Those were some determined voters, who figured out how to vote on it 🙂 #2020-02-11 15:32:59 borkdude lol #2020-02-11 15:35:07 alexmiller also, I as an admin have no ability to change the default - only users can change their own setting. there is another jira about that. #2020-02-11 15:35:50 borkdude maybe we can migrate to clubhouse, I've heard they built it in clojure, so it must be good #2020-02-11 15:38:00 alexmiller I'm done migrating issue data for a while :) #2020-02-11 15:38:55 borkdude just start over. if it's important, the issues will pop up again 😛 #2020-02-11 15:38:57 alexmiller it was nice that they added the new issue, with fewer features, and made it the default, and prevented admins from changing that default #2020-02-11 15:39:22 alexmiller many years ago, I had a boss that literally deleted the issue database with that theory #2020-02-11 15:39:39 borkdude how did it play out? #2020-02-11 15:40:08 alexmiller not great #2020-02-11 15:40:22 borkdude if users have a stake in the issue that was posted, you could also trust on them to migrate the issue, if they found it important #2020-02-11 15:40:50 borkdude (I'm speaking as devil's advocate) #2020-02-11 15:40:58 alexmiller mostly, they just felt like we didn't care about their problems #2020-02-11 15:41:25 alexmiller and we had no way to notify them when we did happen to fix stuff they'd logged earlier #2020-02-11 15:41:43 borkdude hmm, right, if it's customer issues, it might be different #2020-02-11 15:42:29 alexmiller there is some happy medium where you mark things as "won't fix" more aggressively than feels comfortable I think #2020-02-11 14:01:07 borkdude @souenzzo As a workaround, this seems to work:
$ clj -A:lsp:dev -Srepro -Sdeps '{:aliases {:dev {:extra-paths ["a" "b"]} :lsp {:extra-deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}}}' -Spath | tr ':' '\n'
a
b
src
/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/src
/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar #2020-02-11 14:04:01 borkdude This one also works:
$ clj -A:dev -Srepro -Sdeps '{:aliases {:dev {:extra-paths ["a" "b"]}} :deps {clj-kondo/lsp-server {:git/url "" :deps/manifest :deps :deps/root "server" :sha "4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5"}}}' -Spath | tr ':' '\n'
a
b
src
/Users/borkdude/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/Users/borkdude/.gitlibs/libs/clj-kondo/lsp-server/4ba4b40dc47c4e11e76f7a5ecebf7e2d40130fe5/server/src
/Users/borkdude/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar #2020-02-11 20:13:34 hlolli (at)alexmiller, as a continuation of the bugs I mentioned a bit last night related to nixpkgs. It seems as if this isn't related to lumo, but clojure-tools itself. Since I've been tagged a lot on various tickets on nixpkgs due to lumo, I #2020-02-11 20:14:50 ghadi If I understand this correctly, it's a name resolution issue #2020-02-11 20:17:04 hlolli so the problem is that the current clojure, and newer ones for that matter, will by default call maven when called. While a clojure-tools does provide all the classpath needed to run the clojure executeable. So it's basically bootstrapped as it is released. The way to workaround this, would be to force clojure to not check for updates, and just use the bootstrapped jar on runtime for its own environment, and if the user then later wants to resolve dependencies with another deps.edn, it will in those cases ofc continue it's normal behaviour. The issue here is only the runtime environment of clojure and not for another project. #2020-02-11 20:18:44 andy.fingerhut If your deployed application uses a command like java -classpath your:class:path:here and whatever other options you want, then it should not call Maven stuff, unless your application does. #2020-02-11 20:19:22 andy.fingerhut Alex noticed last night that clojure -Scp your:class:path:here might still call Maven and make network connections for that, which he considers a bug, I think. #2020-02-11 20:19:35 hlolli to the default.nix I posted above, I added
doInstallCheck = true;
installCheckPhase = ''
CLJ_CONFIG=$out CLJ_CACHE=$out/libexec $out/bin/clojure -Spath
'';
this fails the same way, maven not found. #2020-02-11 20:20:01 hlolli it can't run itself without internet it seems #2020-02-11 20:20:06 andy.fingerhut Have you tried java command instead? #2020-02-11 20:20:31 andy.fingerhut might require changes to other command line options, too, so not necessarily a one-line change. #2020-02-11 20:20:43 hlolli not sure it matters because clojure is just a shell for calling java anyway #2020-02-11 20:20:47 hlolli #! /nix/store/b85h3rw3y20hyj6s2aqgvlm26a8v61gy-bash-4.4-p23/bin/bash -e
export PATH='/nix/store/rz4aba9s8895b69l3kmx8qb6ir8di2xv-clojure-1.10.1.507/bin:/nix/store/6xxk9hynq5q5nky6wh8l7k4y95y3aksv-rlwrap-0.43/bin:/nix/store/kqr55dy5zjmhklzkkwc85ghf7yxk11qq-openjdk-11.0.6-ga/bin'${PATH:+':'}$PATH
exec -a "$0" "/nix/store/rz4aba9s8895b69l3kmx8qb6ir8di2xv-clojure-1.10.1.507/bin/.clojure-wrapped" " #2020-02-11 20:20:56 andy.fingerhut did you read my message above, completely? #2020-02-11 20:21:09 andy.fingerhut the one that mentioned Alex by name #2020-02-11 20:21:45 hlolli ahh yes ^ #2020-02-11 20:22:05 ghadi does the nixpkg build system intentionally prevent network calls? #2020-02-11 20:22:12 ghadi (I know very little about nix) #2020-02-11 20:22:48 hlolli yes, it allowes network calls, you just need to specify the endpoints by hand. This is by design ("purely functional" buzzwording) #2020-02-11 20:23:09 ghadi makes sense #2020-02-11 20:24:05 andy.fingerhut So it seems you can wait for a bug fix to clojure command, if one is forthcoming, or try java command. #2020-02-11 20:24:08 hlolli just thinking outside the box until a fix arrives, perhaps making some sort of .cpcache file to prevent clojure from attempting to call maven #2020-02-11 20:25:06 andy.fingerhut Sure, if you want to run experiments like that, go for it. If you do not know clojure internals, you are doing black box testing. #2020-02-11 20:26:12 hlolli hence me asking, I'm also ok with solving this on a slow tempo, I'd just remove lumo temporarily in that case #2020-02-11 20:27:18 andy.fingerhut You could also add https://repo1.maven.org to explicit whitelist for nixpkg, if you have such access, and then rinse and repeat as other unexpected host accesses arose. #2020-02-11 20:29:37 hlolli that I can't do, but what I can do, is getting all the jars clojure is attempting to download from nix and put them anywhere visible to clojure. But I just worry if adding -Spath to the clojure wrapper will interfere with normal useage (like two -Spath would be then plausable input parameter to clojure if user provides one and nix one) #2020-02-11 20:30:44 hlolli that is an alternative, but then again, all the files clojure needs to run are already downloaded, so I was hoping to do --offline-mode or something similar until going into more unknown terretory #2020-02-11 20:31:44 andy.fingerhut It might work to run an experiment where you run a command like ps axguwww on the deploy box while your failing clojure command is running to see the full command line and options of all processes, and see what command line options are actually being used. #2020-02-11 20:32:43 hlolli nice, if I find the root of the bug, there's always a possibility of patching (or using forked repo etc..). I'll try this! thanks! #2020-02-11 20:32:44 andy.fingerhut I don't know your time tolerance level for experiments 🙂 #2020-02-11 20:33:09 hlolli I'll write a react app in assembler for fun 😄 #2020-02-11 20:33:58 ghadi haha #2020-02-11 20:38:03 alexmiller the fix in clojure is just extending the flag checks around the tool invocation #2020-02-11 20:38:12 alexmiller (and the equivalent in the windows script) #2020-02-11 20:38:38 alexmiller if someone wants to make a patch, I will likely be doing a release this week and would be happy to include #2020-02-11 20:48:14 hlolli perfect, I think this describes it perfectly #2020-02-11 21:35:38 hlolli just reading trough the shell script quickly, everything makes total sense, and I like how the checksum is based on arguments and environment. Because it's hard to know if the current environment is "stale", so I'd be reluctant to want to make it less strict. Just a simple
if [[ "$stale" = true && "$describe" = false ]]; then
to
if [[ "$stale" = true && "$describe" = false && "$offline" == false ]]; then
or "$dont_recalculate_classpath" instead of $offline #2020-02-11 21:36:12 hlolli > > ...would be enough for my usecase #2020-02-11 21:46:22 alexmiller don't worry about the staleness check, just add it as a flag that affects the running of the tool #2020-02-11 21:47:40 alexmiller this line: if [[ "$stale" = true && "$describe" = false ]]; then #2020-02-11 21:47:59 alexmiller and not -Scp ... #2020-02-11 21:48:06 alexmiller whatever the bash is for that #2020-02-11 21:50:30 hlolli I see, are there any use-cases of people partially wanting to add classpath but otherwise rely on cached (or yet to be cached) classpath? #2020-02-11 21:51:26 hlolli there's extra-classpaths for that I guess, so -Scp would be total opt-out of make-classpaths mechanism #2020-02-11 21:52:55 alexmiller yes, that's it's purpose imo #2020-02-11 21:53:06 alexmiller so should shut off making a cp #2020-02-11 21:53:52 hlolli makes sense to me, with extra-classpaths well documented, shouldn't come as a surprise #2020-02-11 22:38:57 alexmiller are you going to make a patch? #2020-02-11 22:39:34 hlolli yes! be happy to #2020-02-11 22:46:27 andy.fingerhut Wow, just the raw numbers of issues, PRs, and commits on that project are mind-boggling. Sounds like a buzzing hive of activity: https://github.com/NixOS/nixpkgs #2020-02-11 22:55:08 alexmiller looks just like the homebrew core tap :) #2020-02-11 21:48:10 andy.fingerhut When investigating which project.clj files are readable via clojure.edn/read, I have noticed a couple of projects that have both deps.edn and project.clj, and their project.clj is not EDN because it has non-EDN executable Clojure that reads the deps.edn file. I am guessing that if there is both a project.clj and deps.edn file in the root directory of a project's source code, then it doesn't matter whether project.clj is EDN or not? #2020-02-11 21:51:05 andy.fingerhut I've also noticed @U04V15CAJ that you have joined "`#=` club" for several of your project.clj files. I have seen plenty that have an eval-able Clojure expression in place of a version string, that evaluates to a version string, that do not prefix that expression with #= , but simply have the expression. I haven't verified myself whether that actually works, but might be worth testing to see if it does. Either that or plenty of projects left their project.clj files broken in that way, or it only worked in some versions of Leiningen, etc. #2020-02-11 22:05:56 borkdude I think I only used the read-eval because I didn't know any better #2020-02-11 22:15:46 andy.fingerhut There is no problem tooling-wise that I am aware of. This is just me noticing things while checking out ~12,000 project.clj files in projects, to see which are clojure.edn/read'able, which not, and an a few cases they are not, looking at why. Alex Miller mentioned the possibility of having deps.edn use clojure.edn/read for project.clj files if a project had no deps.edn, and I'm still doing the data collection.... #2020-02-11 23:03:22 seancorfield I've never seen #= in a project.clj but I've seen plenty of ~ to evaluate inline expressions... which wouldn't read as EDN, right? #2020-02-11 21:53:36 alexmiller deps.edn is preferred #2020-02-11 21:53:50 alexmiller so if it has that, project.clj won't be used #2020-02-11 22:23:32 andy.fingerhut Thanks for confirmation. Will use that in my summary of findings, and mention it. #2020-02-11 23:07:54 seancorfield Some projects will have both even if project.clj isn't using machinery that reads deps.edn since some people add the latter in response to folks complaining that their projects can't be used by the CLI as git deps 🙂 So they'll start off compatible in terms of general dependencies but may drift apart later if the maintainer forgets to update deps.edn when they update project.clj (in the same way that maintainers of Contrib projects added project.clj files alongside pom.xml files and then the POM was updated but not the project.clj file!). So it's worth mentioning that they could differ but, hopefully, such projects would be a tiny proportion of the whole... #2020-02-11 23:14:19 andy.fingerhut Understood they can differ, in the absence of such automated methods. Right now I'm partly documenting/understanding, and also wanting to make the results as accurate as I know how, by counting a project with both deps.edn and project.clj as "good for tools.deps to read project.clj file, in the last resort" #2020-02-11 23:18:18 andy.fingerhut So I am planning to soon calculate which subset of these projects have each of the 16 possible combinations of files in their repository home directory: project.clj deps.edn build.boot pom.xml. Any others of interest to throw into that mix? #2020-02-11 23:18:48 andy.fingerhut Then I plan to report stats for, within each of those 2^4=16 groups, which ones have EDN-readable project.clj files. #2020-02-11 23:29:00 seancorfield Those are the only four files I can think of that are interesting. It'll be interesting to see which notionally Clojure projects have none of those four (Gradle-based, maybe?). #2020-02-12 00:21:14 alexmiller I'm really concerned most about project that have only a project.clj (and no deps.edn or pom.xml) #2020-02-12 00:24:31 penryu First, wanted to thank @seancorfield for his dot-clojure. 😄
Also wondering if tools.deps has anything similar to leiningen's :injections functionality? #2020-02-12 00:25:02 alexmiller no #2020-02-12 00:26:54 alexmiller instead of running a repl, you could launch your own program that started its own repl that did something similar #2020-02-12 00:27:01 penryu ack. are there any workarounds? something as simple as (require '[clojure.pprint :refer [pp]]) could save a lot of typing over time. #2020-02-12 00:28:19 penryu True. I'm in the process of converting from emacs/cider to vim/conjure. I'll pursue that route. #2020-02-12 00:28:20 penryu Thank you! #2020-02-12 00:29:26 alexmiller like clojure.main effectively just does this:
(repl :init (fn []
(initialize args inits)
(apply require repl-requires)))
#2020-02-12 00:30:11 alexmiller the initialize part will already be handled by the main wrapper, so don't need that line #2020-02-12 00:31:58 alexmiller you'd want to supply the :eval hook: #(do (injections) (eval %)) #2020-02-12 00:32:12 alexmiller something like that, the default is just eval #2020-02-12 00:33:17 penryu ... I'm afraid that's a little beyond my level of knowledge of the repl/clojure.main at the moment, but I'll take some time this week to dig in to how that would work in practice. #2020-02-12 00:33:36 alexmiller that's all the code #2020-02-12 00:33:36 penryu Appreciated, nonetheless! #2020-02-12 00:35:09 penryu the eval hook is defined in deps.edn? #2020-02-12 00:40:03 alexmiller as a command line: #2020-02-12 00:40:07 alexmiller clj -e "(require '[clojure.main :as m]) (m/repl :init #(apply require m/repl-requires) :eval (fn [form] (do (require '[clojure.pprint :refer [pp]]) (eval form))))"
#2020-02-12 00:40:16 alexmiller which you can then fold into an alias if you like #2020-02-12 00:43:28 alexmiller ;; deps.edn:
{:aliases {:repl {:main-opts ["-e" "(require,'[clojure.main,:as,m])(m/repl,:init,#(apply,require,m/repl-requires),:eval,(fn,[form],(do,(require,'[clojure.pprint,:refer,[pp]])(eval,form))))"]}}} #2020-02-12 00:44:51 alexmiller thanks to the Corfield Comma for serving as bash-safe whitespace... :) #2020-02-12 00:43:34 alexmiller then clj -A:repl #2020-02-12 00:44:19 alexmiller you could fold that tiny program into a tool that abstracted out the injections rather than hard-coding them and clean this up a bit, but that's the core of it #2020-02-12 01:07:16 seancorfield Wait... pp is available directly in the REPL started by clj already... #2020-02-12 01:08:08 seancorfield > clj
Clojure 1.10.1
user=> *clojure-version*
{:major 1, :minor 10, :incremental 1, :qualifier nil}
user=> (pp)
{:major 1, :minor 10, :incremental 1, :qualifier nil}
nil
user=>
#2020-02-12 01:09:56 seancorfield @penryu What injections do you want in the Clojure REPL, specifically? (given that pp already seems to be referred in by default) #2020-02-12 01:14:23 alexmiller it's only available in user #2020-02-12 01:14:35 alexmiller injections makes it available in every ns #2020-02-12 01:20:56 kenny fyi #2020-02-12 01:44:06 alexmiller yeah, I know :) #2020-02-12 01:15:14 alexmiller and so I can just point someone to it next time :) #2020-02-12 01:17:40 penryu Thanks for the explanation, @alexmiller! #2020-02-12 01:18:55 penryu @seancorfield might also be something like tools.logging or spyscope (as recommended in a post), though I’m having problems getting either of them to work right now. #2020-02-12 01:20:56 seancorfield (I never used :injections in all the years I used Leiningen and I'm not sure not sure I'd want a bunch of symbols auto-inserted into every namespace while I'm working so...) #2020-02-12 01:44:27 alexmiller well, me neither :) #2020-02-12 01:58:50 penryu I just found out about :injections today, but I'm getting a feel for the [mis]feature parity of deps vs lein. #2020-02-12 02:02:03 alexmiller the philosophy of clj is much more to give you the tools to solve your own problems, not to exhaustively provide every possible feature #2020-02-12 03:14:34 penryu Thanks again for your response. I know leiningen has accumulated a lot of feature-cruft, and I appreciate how clj factors that out. Especially, thank you for the workaround in your blog post! #2020-02-12 01:26:56 penryu Yes, it would need to be sparse in the extreme. But certain things (like pp or #spy) might be reasonable. And I intended to bury it behind an alias (`-A:spy:rebel` or just fold it into rebel) anyway, never in a clean clj session. #2020-02-12 02:05:31 ghadi What is the reason the Corfield comma is necessary in the first place? #2020-02-12 02:24:13 seancorfield Round-tripping to text files and back into bash (and then into Clojure as command-line arguments). #2020-02-12 02:24:31 seancorfield (shell escaping is... facepalm ) #2020-02-12 02:25:56 ghadi is there a technical reason it can not be done with full fidelity, or has no one dived into the GNU Bash Info page? #2020-02-12 02:27:04 ghadi or whatever Windows uses for bash 😂 #2020-02-12 02:37:57 seancorfield @ghadi The "official" clj-on-Windows uses Powershell -- which has different quoting rules from bash (you end up triple-quoting some things -- ugh!) #2020-02-12 02:27:16 ghadi which I think is now... bash #2020-02-12 02:27:37 andy.fingerhut I suspect it could be done with full fidelity, but the prevalence of multiple different versions of bash on Mac OS versus different flavors of Linux adds a level of difficulty to the problem. Also not just spaces, but single and double quotes, dollar signs, braces, ... It can make a grown person weep. #2020-02-12 02:28:58 andy.fingerhut Not sure it is a hill I want to even try to climb, nor die on. #2020-02-12 02:29:11 seancorfield @ghadi I'm sure Alex would welcome patches for it 🙂 #2020-02-12 02:47:53 alexmiller There are patches for it. They are all scary #2020-02-12 02:30:06 seancorfield "It's hard!" is the technical reason I've heard from several people. #2020-02-12 02:30:17 andy.fingerhut I suspect it would be difficult to maintain the shell scripts, too, and lead to copious notes on what works and what doesn't across the platforms. #2020-02-12 02:30:20 ghadi I'm thinking of tools that accept main arguments (like Docker) usually accept a canonical form: an array of strings (which is sort of punting) #2020-02-12 02:30:58 seancorfield The complexity comes from clojure storing all that stuff in text files in .cpcache and then reading it back on subsequent runs. #2020-02-12 02:31:50 andy.fingerhut I know reasonable people might disagree with this statement, but bash is just barely a programming language. #2020-02-12 02:33:26 andy.fingerhut I will end my rant-ish statements on that topic for the evening 🙂 #2020-02-12 02:38:38 andy.fingerhut I lied. As a starting point for consideration, bash --version on the penultimate macOS version (Mojave 10.14) gives this output:
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
Copyright (C) 2007 Free Software Foundation, Inc.
#2020-02-12 02:39:06 andy.fingerhut On Ubuntu 18.04 Linux:
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
#2020-02-12 02:39:16 andy.fingerhut There's 9 years of variation possible right there. #2020-02-12 18:44:31 slipset Also, Catalina wants you over on zsh because the more modern bash have a license that doesn’t please Apple #2020-02-12 02:49:33 alexmiller I actually have a recently raised alternative to the spaces issue that avoids all the escaping that I’m exploring. #2020-02-12 02:50:31 alexmiller Raised by rich and Stu for other purposes but happens to also cover this #2020-02-12 06:29:39 Nate Sutton quoting things for bash is really not all that hard, in my experience, so long as you don't want any shell evaluation. you wrap each thing in ' and then each single quote within it should be replaced with '\'' #2020-02-12 06:30:20 Nate Sutton double quoting introduces a billion corner cases, though #2020-02-12 06:35:00 Nate Sutton I don't really understand how round tripping affects things #2020-02-12 06:35:59 seancorfield @nate_clojurians Then you should write up a patch for clojure and/or tools.deps.alpha that fixes these problems 🙂 #2020-02-12 07:38:34 sogaiu but actually, may be that's a diff issue? #2020-02-12 07:40:32 andy.fingerhut That shows differences in command line parsing for different command shells on Windows. #2020-02-12 07:41:22 andy.fingerhut There are differences between command line parsing for different command shells on Linux/Unix/macOS, too, but bash is a pretty commonly used one, and what clj/clojure scripts are currently written in. #2020-02-12 07:42:34 andy.fingerhut There are things involving single quotes, double quotes, white space, etc. that can be tricky to get correct even in bash, without considering the flavors of Windows command shells described at the link you gave. #2020-02-12 07:44:27 sogaiu i guess if there ever is a cross-platform shell that gets adopted may be this issue will be less of a problem #2020-02-12 07:45:33 andy.fingerhut bash exists on Windows in fairly easily obtainable form. That makes things more common with other platforms, if you use bash on Windows, but it doesn't eliminate the existing difficulties with bash #2020-02-12 07:47:22 sogaiu i use bash on windows via the git for windows installation -- but i hesitate to use it as a shell all of the time.
i think there's a chance that powershell core might at some point be a nicer alternative. #2020-02-12 08:02:45 andy.fingerhut On Linux as well? And macOS? That seems unlikely to me, but stranger things have happened. #2020-02-12 08:06:29 sogaiu i'm not a power bash user but have dealt on and off with shell scripts since the 90s, and though i've historically not been an ms fan, the powershell stuff (at least in core) seemed way more sane to me (though i've used it far less than bash, sh, zsh, etc.) #2020-02-12 08:07:04 sogaiu it's easier to do so when you start over ofc #2020-02-12 08:50:37 penryu At what point is it worth reading code from a file instead of a command line argument? It seems tools.deps has drawn that line in the sand. #2020-02-12 08:55:38 andy.fingerhut I am not sure what you mean by the line in the send. The clojure command lets you give Clojure expressions on the command line after the --eval option (in the init-opts part of the command line args), and/or a file name to evaluate code from named path in the help output by clojure -h #2020-02-12 08:56:01 andy.fingerhut By the line in the sand, do you mean it lets you pick what you want? #2020-02-12 18:09:37 seancorfield Caching. clojure does two passes -- and invokes the JVM twice -- if the classpath/main/JVM options are not cached (in a file). It does just one pass with the cache in place. That's a big deal. #2020-02-12 14:46:59 sparkofreason Expanded version of my earlier question about javac: suppose I want to get in between the resolve-deps and make-classpath steps before running clj, say to do some other processing on files fetched from git, add resulting artifacts to the classpath. Are there any examples of tools (or whatever) doing such? #2020-02-12 15:11:27 delaguardo You can try https://github.com/EwenG/badigeon
I’m not sure this is exact solution for your specific task, but could provide some missing tool’s parts #2020-02-12 15:13:21 sparkofreason Yes, assuming nothing else existed, that was going to be my starting point. Very useful stuff. #2020-02-12 14:55:17 dominicm Not that I'm aware of, that's a new, separate tool. #2020-02-12 14:55:42 dominicm However, I think that would be really useful for the javac case. #2020-02-12 14:56:03 dominicm Other tools are generally shell scripts which you run first, but that reduces the smooth integration with cider, etc. #2020-02-12 18:40:52 andy.fingerhut OK, I do not have stats weighted by number of http://Clojars.org project downloads right now, but I do have some stats showing that out of 11,736 Github projects I successfully cloned, and that is the subset of them that have a project.clj file but no deps.edn file in their root directory, 91.8% of those project.clj files were readable using clojure.edn/read , and had their defproject form first, and a reasonable-looking value for the :dependencies key (or no dependencies key, which is completely correct and fairly common). More data and steps I used to collect it here: https://github.com/jafingerhut/haironfire #2020-02-12 18:42:36 alexmiller cool (and nice repo name :) #2020-02-12 18:42:47 andy.fingerhut The most common reason that a project.clj file threw an exception while attempting to use clojure.edn/read to read its contents: regex literals are not valid EDN. Those are reasonably common in project.clj files, for things like jar inclusions/exclusions, or other kinds of file name selection sets. #2020-02-12 18:43:52 alexmiller that's kind of interesting #2020-02-12 18:44:09 alexmiller I wonder if you could hack the edn reader to read those into tagged literals or something #2020-02-12 18:44:54 alexmiller how much additional % does that cover? #2020-02-12 18:46:02 andy.fingerhut Good question, which I thought of attempting, but haven't yet. That was the first thing in the project.clj file that caused clojure.edn/read to throw, and there could be others after that, which would only be straightforward to calculate with such a modified version of clojure.edn/read #2020-02-12 18:47:27 alexmiller such a thing could possibly even make its way into clojure's real edn reader #2020-02-12 18:49:28 ghadi regexes aren't portable though #2020-02-12 18:49:43 alexmiller I didn't say make it into edn #2020-02-12 18:50:05 alexmiller there are lots of things the clojure edn reader does beyond the edn spec #2020-02-12 18:50:25 ghadi hmmm.jpg #2020-02-12 18:50:37 alexmiller and I'm not suggesting reading it into java pattern, but into a tagged literal #2020-02-12 18:50:44 ghadi same thing for the var dispatch I was talking to you about #2020-02-12 18:50:48 alexmiller #regex or whatever #2020-02-12 18:51:14 alexmiller less sure on vars if that makes sense as you're then connecting to code #2020-02-12 18:53:33 ghadi (tangent, but I want to have the choice of whether to connect to code or read as tagged) #2020-02-12 19:00:52 dominicm Edamame can read regex I think #2020-02-12 19:04:53 andy.fingerhut Based on the numbers I have seen, if the project.clj files that threw exception due to regex are edn-readable in all other ways except the regex literals, I think the most the numbers above could improve would be from current 91.8% up to close to 95% or 96%, but that would be noticeable, I think. #2020-02-12 19:06:07 andy.fingerhut Sounds like a straightforward thing to hack up locally on my system. I will let you know if I try it out. If anyone hacks up a version of clojure.edn/read before I do, let me know. #2020-02-12 19:08:54 ghadi @andy.fingerhut you could switch to tools.reader and monkey patch that to read regex + vars #2020-02-12 19:09:20 andy.fingerhut Cool, that may be the quickest way. #2020-02-12 19:09:48 ghadi naughty, but straightforward #2020-02-12 19:09:48 andy.fingerhut Maybe I should take the opportunity to see if clojure.edn/read and tools.reader edn reader behave differently on any of these 11,736 files 🙂 #2020-02-12 21:06:16 alexmiller this is about existing data in project.clj files, so I don't think that helps anything #2020-02-12 21:07:37 sogaiu you're talking about converting regexes as well as whether the result is valid edn, no? #2020-02-12 21:08:45 sogaiu anyway, for future cases, it could be useful for folks to express their regex using regal and having a file that is more likely to be edn? #2020-02-12 21:23:52 andy.fingerhut This is experimental exploration here, wondering how we might use a secure safe reader, either existing, or perhaps slightly tweaked, to read existing project.clj files. #2020-02-12 21:24:16 andy.fingerhut I doubt we could ever convince hundreds of people to update the project.clj files to use a new regex representation, for something that already works for them today. #2020-02-12 21:57:16 sogaiu sure -- just musing about future possibilities. sorry for the noise. #2020-02-12 22:03:25 andy.fingerhut No worries. Just hoping the context makes it clear why we're talking like we do 🙂 #2020-02-12 22:43:35 sogaiu thanks for your ongoing patience and explanatory efforts 🙂 #2020-02-12 21:47:02 andy.fingerhut Fraction of the 11,736 project.clj files that become 'good' in my experiments (readable using clojure.edn/read without throwing exceptions, defproject form is first, value of dependencies key looks reasonable) goes up from 91.8% to 95.7% with a hacked up version of EdnReader class that can read regex literals. Details here: https://github.com/jafingerhut/haironfire/blob/master/README-extended-edn-reader.md #2020-02-12 21:52:53 andy.fingerhut As mentioned in #clojure-dev channel discussion, my 'good' evaluation here currently has no checking for whether the dependencies list is actually the one used by the project, e.g. no checking for or evaluating the effects of Leiningen plugins, e.g. Lein-modules. There are also approximately another 1,000 git repos associated with Clojars.arg artifacts that have no project.clj nor deps.edn files in their home directory. Most of those probably have some kind of hierarchical structure to their files, with one or more independent Leiningen projects in the same source repo. I haven't tried analyzing those. #2020-02-12 22:10:55 Nate Sutton I wonder if you could use sci to safely figure out what exactly is not valid edn? #2020-02-12 22:13:55 Nate Sutton I mean, beyond the regex literals #2020-02-12 22:13:55 Nate Sutton I mean, beyond the regex literals #2020-02-12 22:18:52 andy.fingerhut Short answer appears to be "in-line eval'able code inside the defproject form" #2020-02-12 22:19:38 Nate Sutton ahhhhhh #2020-02-12 22:19:39 Nate Sutton cool #2020-02-12 22:19:39 andy.fingerhut One could pretty easily extend an EDN-style reader to read those things, too, but I suspect the return-on-time-invested is getting fairly low. #2020-02-12 22:26:20 Nate Sutton yeah for sure, definitely diminishing returns at this point, from the looks of it #2020-02-12 22:26:41 Nate Sutton I just saw the need for a safe, custom edn reader and wanted to share 🙂 #2020-02-12 22:56:44 borkdude > One could pretty easily extend an EDN-style reader to read those things, too, but I suspect the return-on-time-invested is getting fairly low.
@andy.fingerhut That's exactly what edamame already is. #2020-02-12 23:00:18 borkdude e.g.:
(parse-string "#=(+ 1 2 3)" {:read-eval true})
;;=> (read-eval (+ 1 2 3)) #2020-02-12 23:00:44 borkdude but :read-eval can also be a function that transforms the value in any way you want #2020-02-12 23:01:52 andy.fingerhut Understood. Part of the context of this discussion, I believe, is what dependencies outside of itself and Clojure that Alex is willing to take on for tools.deps. #2020-02-12 23:02:06 andy.fingerhut I do not know his detailed thoughts on the matter, other than some nuanced approximation of "as few as possible". #2020-02-12 23:03:23 borkdude Yeah, I understand. I guess you could also have a "forgiving" EDN parser that just skips over what it cannot read. If that leaves a valid :dependencies vector, that's probably all that counts. #2020-02-12 23:05:00 andy.fingerhut I suspect the harder part of getting a valid dependencies sequence (one thing I learned is Leiningen accepts, and a few project.clj files actually use, lists or even a map for the value of :dependencies ), is the existence of git repos with a project.clj file in the root directory, and then several more in subdirectories, with interactions between their contents, e.g. via Lein-modules plugin or other mechanisms. #2020-02-12 23:05:05 borkdude so what is the summary of your research, what's the percentage of EDN-readable project.cljs? #2020-02-12 23:06:57 andy.fingerhut 91.8% of 11,736 project.clj files that are in the root directory of git repos, and those were selected because they do not also have a deps.edn file in the root dir, are clojure.edn/read readable in Clojure 1.10.1. If you extended clojure.edn/read to not throw an exception on encountering a regex literal, it goes up to 95.7% #2020-02-12 23:08:01 andy.fingerhut But as I mentioned two comments ago, the percentage that are EDN-readable isn't all that one cares about if one wants to get correct dependencies from a Leiningen source repo. #2020-02-12 23:08:05 borkdude very cool. I assume you communicated this summary to Alex as well? #2020-02-12 23:08:16 andy.fingerhut Earlier today in this same channel. #2020-02-12 23:08:32 borkdude true, but maybe those multi-project.clj repos are also in the <10% range? #2020-02-12 23:08:56 andy.fingerhut I haven't tried to determine that yet. #2020-02-12 23:09:28 borkdude I haven't had time to read slack today. Nice progress! #2020-02-12 23:10:02 andy.fingerhut Sure, it's been fun hacking away. Also makes me realize that the actual question is harder to answer 🙂 #2020-02-13 05:45:43 rberger Does seancorfield/clj-new have a way to specify copying a binary file from the template source dir to the target dir? The doc kind of says that the render function without a data argument would do that, but it looks like it treats the file as text with linefeeds and corrupts the binary data. Is there a way to do this that still fits into the ->files body? #2020-02-13 17:32:51 seancorfield @U07GPFFAS That sounds like a nice enhancement! Feel free to open a GitHub issue. Do you know if either lein-new or boot-new support such a thing? (`boot-new` was originally based on lein-new and clj-new was forked from boot-new but I haven't tracked their progress much since). #2020-02-13 18:01:05 rberger I haven't used the others {lein,boot}-new I'll open a github issue. I guess in the short term I could create another renderer for binary files that would do the copy #2020-02-13 18:03:51 seancorfield Feel free to provide a PR with that enhancement too. #2020-02-13 18:08:25 seancorfield @U07GPFFAS Just looked in the code and there's a raw-resourcer which I suspect can be used to copy binary files. I can't say I remember trying that but it's worth a shot. #2020-02-13 18:09:11 rberger It wasn't clear to me when I looked at that, but will look at it today. Thanks! #2020-02-13 21:25:44 seancorfield Thanks for the heads up. Weird error Failed to read artifact descriptor for com.fasterxml.jackson.core:jackson-databind:jar:2.7.5 #2020-02-13 21:26:30 seancorfield Hmm, it's leftover in the pom.xml by accident 😞 #2020-02-13 21:41:07 seancorfield I've pushed 0.9.0 with that fix but the doc system build process is broken right now (by changes to Clojars CDN setup). Once that's fixed, I'll rebuild the clj-new docs. #2020-02-14 02:01:31 rberger Thanks! #2020-02-13 13:25:13 Andreas S. Hello I try to run clj on windows in cygwin but I get: #2020-02-13 13:25:16 Andreas S. $ clojure
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main #2020-02-13 13:25:21 Andreas S. Any ideas? #2020-02-13 13:33:20 delaguardo try clojure -Spath you should get a string with all the jars including clojure-%version%.jar then go check if this jar is actualy available #2020-02-13 13:36:13 Andreas S. If I run "clojure -Spath" I get the same error #2020-02-13 13:36:53 borkdude @andreas.scheinert We had someone here a while ago who has an older version of clojure on his path. What do you see when you type clojure -h #2020-02-13 13:37:05 Andreas S. I have a file here: /usr/local/lib/clojure/libexec ==> clojure-tools-1.10.1.507.jar* #2020-02-13 13:38:21 Andreas S. borkdude here is version "clojure -h" print out #2020-02-13 13:38:23 Andreas S. Version: 1.10.1.507 #2020-02-13 13:38:34 borkdude that's good then #2020-02-13 13:40:30 Andreas S. I know its a bit side stepping but, ok I took the effort and installed it on powershell #2020-02-13 13:40:47 Andreas S. clj and clojure give me now a REPL #2020-02-13 13:40:49 Andreas S. but #2020-02-13 13:41:25 Andreas S. clj -Sdeps '{:deps {seancorfield/clj-new {:mvn/version "0.8.6"}}}' \
-m clj-new.create \
\
appcompany.funapp
#2020-02-13 13:42:31 Andreas S. I figured I remove all the \ and create a long single line of command, but then that still gives me: #2020-02-13 13:42:32 Andreas S. Error while parsing option "--config-data {:deps {seancorfield/clj-new {:mvn/version 0.8.6}}}": java.lang.NumberFormatException: Invalid number: 0.8.6 #2020-02-13 17:36:28 dominicm I notice that resolve-deps doesn't mention that it will be concurrent even if :threads isn't provided. Should it? #2020-02-13 17:40:40 seancorfield Even following that Windows guide didn't work for me 😞 #2020-02-13 17:41:18 dominicm make-classpath-roots mentions the system separator, but it doesn't really make sense in the context. #2020-02-13 17:42:32 seancorfield @andreas.scheinert This worked for me:
> clj -Sdeps '{:deps {seancorfield/clj-new {:mvn/version ""0.8.6""}}}' -m clj-new.create appcompany.funapp
So, double "" inside ' #2020-02-13 17:47:19 seancorfield (that said, even with Python 3.7 installed from the Windows Store, I can't actually get the new project to run but that should be taken up in #libpython-clj at this point) #2020-02-14 11:42:50 Andreas S. Thank you sean for replying! Yes the quoting did the trick but I am stuck kat the same point as you I couldn't get windows python to work with libpython-clj #2020-02-13 20:33:03 sam For private git repo deps on Windows, is only Pageant supported as the ssh agent for authentication or can OpenSSH also work? #2020-02-13 21:03:16 alexmiller try it and see? #2020-02-13 21:03:38 alexmiller I know that pageant works, don't know about openssh #2020-02-13 21:13:30 sam @alexmiller I have been trying. It seems to be attempting to use OpenSSH. I can tell because the specific error message I get changes depending on whether github is added to .ssh/known_hosts. However, even though I can use git on Powershell to clone a private git repo, I get an error when trying to run clj with a private github dep. The lowest part of the stack trace is just:
Caused by: com.jcraft.jsch.JSchException: Auth fail
at com.jcraft.jsch.Session.connect(Session.java:519)
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:126)
... 45 more
which doesn’t give me a ton to go on. It’s been really painful to debug. #2020-02-13 21:13:57 alexmiller well I think I would say, no it doesn't work then #2020-02-13 21:14:36 sam It’s been finicky to get it working even with Pageant. So I’m never sure whether something just doesn’t work, or if I’m just doing something slightly wrong #2020-02-13 21:14:38 alexmiller ssh is generally tricky, particularly with access to the wide world of credential providers #2020-02-13 21:16:03 alexmiller there are some options (including shelling out to just use git) which we have explored. newer versions of jgit use apache mina instead of jsch. I have spent some time working on evaluating these alternatives but haven't had time to really get to the end of that process #2020-02-13 21:16:29 alexmiller another option would be to clone, then use as a local repo instead #2020-02-13 21:17:10 alexmiller or to clone, build and install a local artifact, then use that #2020-02-13 21:20:09 sam Okay, thanks. One other question for you. I’ve resorted to trying OpenSSH on one of our dev’s windows machines only because we couldn’t get deps working with Pageant on that machine. I have no idea why it’s not working because we’ve been able to get it working on other Windows machines. Based on your experience, do you have any idea of any “gotchas” we might be running into? The error message is the same as above. #2020-02-13 21:21:05 sam If not, that’s okay, I know that’s a very vague question. Just wondering if anything comes to mind of the top of your head #2020-02-13 21:22:57 alexmiller I don't unfortunately. the jsch stuff is a bit flaky #2020-02-13 21:26:23 sam Okay, well thanks anyway. Btw, just wanted to say I was at StrangeLoop in September and really enjoyed it. Thanks for doing that. #2020-02-13 21:27:03 alexmiller great! thx #2020-02-13 21:24:53 ghadi if you're using any ssh agent, make sure the SSH_AUTH_SOCK variable is set so that Clojure (via JSch) knows where the agent is #2020-02-13 21:27:15 sam cool, thank you, I’ll look into that. #2020-02-13 22:01:48 sam Are you sure this is applicable for Windows? I can’t find much information about it in the context of Windows, and Clojure seems to work fine on my windows machine using Pageant as the ssh-agent without this env variable being set. #2020-02-13 22:03:56 ghadi ssh clients discover where an agent is running by consulting that environment variable #2020-02-13 22:04:52 ghadi that's what jsch does, too.... sorry Windows just adds a lot of extra wildcards because of all the various linux-y environments #2020-02-13 22:04:57 ghadi paging @U04V70XH6 #2020-02-13 22:05:04 ghadi just in case he knows #2020-02-13 22:11:49 seancorfield I know nothing about ssh on windows, sorry #2020-02-13 22:12:57 seancorfield I only ever use ssh stuff via Ubuntu / WSL on windows 😁 #2020-02-13 22:15:00 sam Thanks anyway! #2020-02-14 00:30:34 andy.fingerhut I've added one section to the README for my haironfire repo here: https://github.com/jafingerhut/haironfire#some-leiningen-projects-can-depend-upon-multiple-files-andor-tools-to-determine-the-actual-dependencies. merely mentioning some challenges of calculating accurate dependencies for Leiningen projects, which I have not attempted to quantify in my work how often those tricky cases even arise in the projects I've analyzed. I don't have any plans to try to analyze those things, but others are certainly welcome to if they are interested, using any or all of the code in that repo if it helps. Perhaps the biggest hole in the 91.8% and 95.6% numbers I reported for EDN-readable project.clj files are: (a) Those were for repos with a project.clj file in the home directory. Outside of the 100% number is about another 10% of repos that have no project.clj nor deps.edn file in the home directory, many of which probably contain project.clj files in subdirectories that I haven't attempted to find or read. And (b) the difficulties in determining dependencies even after you read a single project.clj file, if there are tools like Lein-modules or similar things in use, again which I haven't tried to determine which projects might use such things. #2020-02-14 00:32:54 andy.fingerhut Or in summary, I will now more fully understand if the decision is made not to have tools.deps read project.clj files 🙂 #2020-02-14 07:53:16 dominicm Nothing to do with git, this equivalence comparison just already exists. So my changes don't help. I could have sworn I tested it... #2020-02-14 07:54:35 dominicm Oh, I did. But I was being very, very stupid (= (:local/root coord-x) (:local/root coord-x)) comparing x to x 🙂 #2020-02-14 18:42:25 dominicm I wrote a project to visualize the relationship between your project and it's dependencies. It currently relies on the deps.edn format in order to function. Here's the project: https://github.com/SevereOverfl0w/vizns #2020-02-14 18:42:25 dominicm I wrote a project to visualize the relationship between your project and it's dependencies. It currently relies on the deps.edn format in order to function. Here's the project: https://github.com/SevereOverfl0w/vizns #2020-02-14 19:33:06 borkdude done #2020-02-14 19:54:44 dominicm Thank you 😚 #2020-02-17 11:33:27 teodorlu This is really cool! Fantastic way to confirm that your code is all spaghetti after all.
Some quick notes on usability:
• I first naïvely ran it in a folder without a deps.edn file, and spent some time figuring out why it didn't work.
• Then I reran it in an actual project, only to find out I was making it crash because I hadn't supplied any main args
• Then I figured I needed to sudo apt install graphviz, because of the dependency on dot.
Then all was fine! #2020-02-17 12:21:36 dominicm I don't have the headspace to pick those up yet, but I'd welcome a PR if you have time. Thanks for the feedback, I appreciate it! #2020-02-14 20:14:42 alexmiller @dominicm the readme example has tools.deps.graph for the main class #2020-02-14 20:36:52 dominicm D'oh, used that as inspiration. I thought I'd got it all, will fix, thanks. #2020-02-14 20:16:36 alexmiller I assume that should be io.dominic.vizns.core #2020-02-14 20:18:00 alexmiller also, it looks like you happened to rely on make-classpath-roots which I added in tda 0.8.666 (which I never announced) then made public in the release earlier today #2020-02-14 20:18:32 alexmiller I didn't expect anyone would make use of it in interim :) #2020-02-14 20:19:50 alexmiller seems like you could have just used make-classpath though? #2020-02-14 20:49:01 dominicm I could have split the string, yes. It was nice to think of all the saved cycles of not doing that. #2020-02-14 20:52:22 alexmiller I sent you a pr #2020-02-14 21:00:08 dominicm Oh, that's a better solution. Thanks for the PRs, makes it mergeable from my phone. The final one won't merge at the moment, but I can resolve from a keyboard :) #2020-02-14 20:20:27 alexmiller there's some ongoing discussion between rich and I in that area about structured classpath data and I'm not ready to commit to anything there yet #2020-02-14 20:21:13 borkdude maybe the tool should just deal with a classpath as input so users can decide which tool they use to create it (`clojure -Spath`, lein classpath, etc)? #2020-02-14 20:50:20 dominicm I want to do that, but then I have to identify classpath entries back to their library name. #2020-02-14 20:51:49 dominicm I'd need something slightly different to a classpath. Haven't figured out what that looks like yet from an agnostic point of view. #2020-02-14 20:55:22 dominicm Git dependencies are pretty difficult to name. #2020-02-14 21:04:05 borkdude right! #2020-02-14 20:21:39 borkdude (I haven't looked at any details so I don't know if that's going to make sense) #2020-02-14 20:32:08 borkdude for the project.clj git dep problem, one maybe silly but effective solution could be to let tools.deps clone the project, then let it invoke lein classpath on that project, so lein can deal with itself, and write that classpath to a file which is cached, so next time tools.deps wants to know the classpath that that dep is going to produce, it doesn't have to invoke lein again #2020-02-14 20:33:34 borkdude haven't really thought this through, feel free to dismiss it. have a great weekend. 🍻 #2020-02-14 20:35:19 alexmiller I think it's interesting to consider a dep type that could provide it's deps explicitly like that #2020-02-14 20:35:35 alexmiller hadn't thought about it before #2020-02-14 20:48:16 andy.fingerhut That still requires eval-level trust in projects you depend upon, at dep resolution time, yes? I do not know if it is any kind of clear bright line to avoid that -- just asking to confirm that the proposed idea still does do eval of project.clj files, even if you never require their code. #2020-02-14 21:02:37 borkdude true, it will eval. it could potentially be a solution for lein projects with java sources as well btw. #2020-02-14 21:04:38 borkdude I guess the user of a git dep can check the project at given sha if it's deemed safe #2020-02-14 21:06:44 borkdude Running library code from the wild is going to be eval'ed anyway, so you should in theory always be checking the code of a dep before you run it? #2020-02-14 21:15:54 andy.fingerhut It would be safest, yes. There is a lot of web of trust kinds of things going on, for sure, and the common practice that if something is discovered bad, it will be publicized. That all works with your idea, too, still. I'm only perhaps nit-picking that deps resolution time is another eval-time opening this creates, over and above require'ing the code and running it. To take advantage of it, a malicious party still needs to become trusted, i.e. depended upon by other projects, so in that sense it isn't a new opportunity for attack. #2020-02-15 09:38:31 penryu I'm used to seeing maven dep versions in deps.edn as {:mvn/version "0.4.490"}}, but I've found one that specifies deps with #:mvn{:version "0.4.490"}. Looks like a reader macro, but I haven't seen it before. #2020-02-15 09:39:03 borkdude that's called a namespaced map #2020-02-15 09:40:09 penryu it looks like it's equivalent to the other syntax; are they interchangeable? #2020-02-15 09:41:49 penryu Ah, you can use it to factor out a common namespace in map keys? #2020-02-15 09:43:02 penryu That's exactly what I was missing. Thanks, @borkdude. #2020-02-15 17:09:22 alexmiller Yep #2020-02-15 17:10:04 borkdude done #2020-02-15 17:13:57 benedek done #2020-02-16 11:07:57 kwladyka ~/L/M/c/s/P/l/pdf-converter clj -Aoutdated
| Dependency | Current | Latest |
|---------------------------+----------+----------|
| org.clojure/clojure | 1.10.0 | 1.10.1 |
| org.clojure/clojurescript | 1.10.520 | 1.10.597 |
! ~/L/M/c/s/P/l/pdf-converter clj -Aoutdated -u
Updating: deps.edn
org.clojure/clojure {:mvn/version "1.10.0"} -> {:mvn/version "1.10.1"}
org.clojure/test.check {:mvn/version "0.9.0"} -> {:mvn/version "0.10.0"}
com.cognitect/test-runner {:sha "028a6d41ac9ac5d5c405dfc38e4da6b4cc1255d5"} -> {:sha "f7ef16dc3b8332b0d77bc0274578ad5270fbfedd"}
~/L/M/c/s/P/l/pdf-converter clj -Aoutdated
| Dependency | Current | Latest |
|---------------------------+----------+----------|
| org.clojure/clojurescript | 1.10.520 | 1.10.597 |
~/L/M/c/s/P/l/pdf-converter #2020-02-16 11:08:12 kwladyka What is the logic here? #2020-02-16 11:08:48 kwladyka {:paths ["resources" "src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}}
:aliases {:test {:extra-paths ["test"]
:extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}
me.raynes/fs {:mvn/version "1.4.6"}}}}
:runner {:extra-deps {com.cognitect/test-runner
{:git/url ""
:sha "f7ef16dc3b8332b0d77bc0274578ad5270fbfedd"}}
:main-opts ["-m" "cognitect.test-runner"
"-d" "test"]}}
deps.edn #2020-02-16 11:16:01 kwladyka it is showing something different to update and update something different #2020-02-16 11:16:07 kwladyka and what cljs doing here? #2020-02-16 11:37:53 dominicm Maybe the first one is looking at transitive dependencies when it shouldn't be? #2020-02-16 11:38:41 kwladyka What do you use for updates? I found there is also vulcan #2020-02-16 11:46:57 dominicm I don't use anything :) #2020-02-16 11:51:34 borkdude (for maven deps) #2020-02-16 18:17:43 seancorfield I would never run a tool that auto-updated my deps. I used Depot for a while but the 2.x version doesn't work with our setup (because it processes deps files directly instead of relying on t.d.a. to process all the aliases and override etc via a non-user deps.edn specified via CLJ_CONFIG). #2020-02-16 18:19:34 seancorfield Like @borkdude I created a script instead -- it runs clojure -Stree and massages the output, then creates a deps.edn with all the :mvn/version deps changed to "RELEASE" and re-run clojure -Stree against that, and then compare the top N levels of those trees. #2020-02-17 10:00:03 Michaël Salihi This script is available on your Github ? #2020-02-17 10:02:03 borkdude I made it based on the idea of Sean #2020-02-17 10:17:33 Michaël Salihi @borkdudeNice script, to use and also for learning beautiful Clojure scripting.
Babashka rocks! Thank you! #2020-02-17 10:23:12 borkdude Thanks 😊 #2020-02-17 17:56:31 seancorfield @UFBL6R4P3 Our script at work is somewhat proprietary and heavily tied to how our monorepo works (30 subprojects, each with their own deps.edn and with an overall deps.edn that is used instead of the ~/.clojure user-level one, via the CLJ_CONFIG environment variable). #2020-02-17 18:45:25 Michaël Salihi @U04V70XH6 I understand. Thx for the information, I don't know CLJ_CONFIG.
New thing in my baggage! #2020-02-17 07:13:08 benedek a tool that creates a PR automatically would be nice i think. added benefit is all the tests run if you have some autobuild set up. such tools exist for other ecosystems i think #2020-02-19 10:08:20 souenzzo From
https://github.com/portkey-cloud/cfn-yaml/issues/1
How I supposed to import a project withtout a clear ArtifactID?
Consider:
I publish my/lib {:rev :v1} that requires cfn-yaml {:git/url "" :sha v1}
Someone else import my/lib {:rev :v1} portkey-cloud/cfn-yaml {:git/url "" :sha v2}
It will result in #2020-02-19 13:37:25 alexmiller To get comparisons, you need to use consistent ids #2020-02-19 13:38:13 alexmiller This is admittedly challenging with git deps, and they work best with things like tools that are edges of a dependency graph #2020-02-19 13:57:40 dominicm I generally strongly recommend a id in the readme #2020-02-19 13:59:27 alexmiller Seems like a good practice #2020-02-19 15:03:32 alexmiller it kind of is with the advice of
github-yourname/time-lib
#2020-02-19 14:15:02 dominicm If repos were the canonical source of what id they represented, that would solve this. It would also make the deps.edn format very different. #2020-02-19 14:18:53 alexmiller it would also break forking use cases etc #2020-02-19 14:19:31 alexmiller it was long considered #2020-02-21 08:50:40 teodorlu Hey! 👋
Any idea on why this doesn't work?
$ clj -e '(doc ns-map)'
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: doc in this context
Full report at:
/tmp/clojure-135646063998456412.edn
#2020-02-21 08:58:49 dominicm You need to require clojure.repl, doc is defined there, not in core #2020-02-21 08:59:06 dominicm user.clj requires clojure.repl for you #2020-02-21 13:54:24 alexmiller No, Clojure.main does this in the repl setup #2020-02-21 09:00:59 vlaaad I would guess it would be more correct to say that starting repl requires clojure.repl for you #2020-02-21 09:06:18 dominicm Quite right, I'm not sure why I thought otherwise. Brain fart. #2020-02-21 13:14:45 Wes Hall Making some increasingly half-hearted attempt to get DynamoDBLocal running to start embedded for tests, but it seems to depend on sqllite4java which exists in the mvn repos but with <packaging>dylib</packaging>. Am I right in assuming that tools-deps doesn't have support for downloading dylib dependencies? #2020-02-21 13:48:19 kszabo I prefer to use docker-compose to set up local service dependencies:
version: '2'
services:
dynamodb-local:
image: 'dwmkerr/dynamodb' #2020-02-21 13:49:11 kszabo but this is not related to #tools-deps this way #2020-02-21 14:20:25 Wes Hall @thenonameguy Yeah, I will probably move to something like that. Dynamo local, at least in theory, does support embedded running so should be possible to throw up an in-memory version in a text fixture and tear it down afterwards, but the dependency on sqllite (and thus native libraries), is a bit of a hurdle. I'll probably give up, but just thought I would check that I am not missing a route I could at least try, but I think the docker-compose option is probably the way to go. Thanks. #2020-02-21 14:21:11 Wes Hall Downloading native (thus platform dependent) libraries, is a bit awful anyway, I wouldn't be surprised if it's not supported. #2020-02-21 14:33:47 teodorlu @vlaaad @dominicm @alexmiller thanks for the help clearing that up! I realize why doc shouldn't work above, and I get the expected result when I require it as I "should" do.
$ clj -e "(require '[clojure.repl :refer [doc]]) (doc ns-map)"
-------------------------
clojure.core/ns-map
([ns])
Returns a map of all the mappings for the namespace.
#2020-02-21 17:14:48 rickmoynihan Not sure if this is a known regression or not… but I was using some private S3 buckets as mvn repos; with a URL of the form s3p://<bucket>/<directory>/
I’ve been using this for years (previously with leiningen) and found earlier in the week that I could no longer resolve deps via the clj tool.
I suspect this was caused by the recent changes in t.d.a to using the cognitect aws lib; but I can’t be certain.
Anyway I’ve just tried changing the bucket URL’s to be of the form s3:// and it now works. No idea what the difference is in the protocols; but I’ve seen both forms over the years. #2020-02-21 17:17:21 gerred s3p is for hadoop/presto it seems. #2020-02-21 17:18:02 gerred not an official protocol. AWS has a few other protocols for similar reasons (with EMR specially): s3a and s3n come to mind. #2020-02-21 17:18:02 gerred not an official protocol. AWS has a few other protocols for similar reasons (with EMR specially): s3a and s3n come to mind. #2020-02-21 17:19:58 gerred so maybe this was only a wagon thing. #2020-02-21 17:21:48 gerred it looks like it's even gone from newer versions of Wagon #2020-02-21 17:22:18 rickmoynihan yeah iirc tools.deps used the s3 wagon until recently #2020-02-21 17:23:04 gerred oh no it's still there #2020-02-21 17:23:05 gerred yeah #2020-02-21 17:27:28 dominicm Alex did ask to know if anything broke. I think compatibility was the intention. #2020-02-21 17:33:46 rickmoynihan Yeah that’s why I’m mentioning it.
The (end user) fix is trivial (just change to use s3://) once you know what the problem is though. #2020-02-21 17:53:50 alexmiller s3p has never been a documented supported protocol for clj #2020-02-21 17:54:23 alexmiller So was working before by accident, not by intent #2020-02-21 17:54:49 alexmiller s3p is an invention of the spring aws wagon #2020-02-21 17:55:06 rickmoynihan Ah interesting, I think I just copied my URLs from lein and didn’t think anything of it when they worked #2020-02-21 17:55:34 rickmoynihan Anyway it’s not a problem, I just thought I’d mention it incase it happens to others #2020-02-24 05:54:50 onetom I'm trying to switch to Google's Maven Central mirror.
I've added a mirror section to my ~/.m2/settings.xml but I'm not sure if it's used by clj.
How can I check what actual URL is it downloading from when it says:
Downloading: org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar from central
I've tried -Sverbose and -Sdescribe but neither of those options provide an answer #2020-02-24 13:52:07 alexmiller I don’t think there is any way to see that, but there is code with the intent to use mirrors in settings #2020-02-24 13:52:42 alexmiller Someone else from Asia walked down this same path a few months ago and it was working #2020-02-24 06:00:33 EmmanuelOga perhaps you need to also add it to the deps.edn map? #2020-02-24 06:01:35 onetom I'm trying to move it out from my deps.edns becaues mirror settings are not project specific #2020-02-24 06:02:14 EmmanuelOga what if you put it in the global deps.edn? #2020-02-24 06:02:46 EmmanuelOga $HOME/.clojure/deps.edn #2020-02-24 06:04:14 onetom then pure maven tools wouldn't benefit from this setting #2020-02-24 06:04:52 onetom plus im not sure what's the semantic difference between overriding the repo called "central" vs specifying mirrors instead #2020-02-24 06:06:36 onetom it's also a bit unclear how the :mvn/repos key is merged into the hardwired default values, because i haven't found a way to print the effective value of it using clj #2020-02-24 06:08:27 onetom i had the impression that the :mvn/repos key in my project's deps.edn overrides the value coming from ~/.clojure/deps.edn but i haven't found this being documented #2020-02-24 06:14:46 onetom the only reason i suspect that the <mirror> setting is taken into account is because the installation of my deps from scratch takes 34s instead of 51s #2020-02-24 06:34:57 EmmanuelOga heh nice #2020-02-24 06:36:03 EmmanuelOga are you working with a setup that for some reason needs toperform installs from scratch often? #2020-02-24 07:02:11 onetom im in hong kong and im on crappy mobile connections often, so using mirrors make a big difference even if im not doing from-scratch installs.
on the other hand im also experimenting with building self-contained developer workstation setups and then i want to test from-scratch builds sometimes just to make sure everything still works.
a datomic upgrade also pulls in quite some dependencies and our projects might be on slightly different versions of common libraries and all this adds up...
just to demonstrate the absolute scale of problems:
• im taking a 30minute ferry ride twice a day to work and i could spend about 20minutes of that working
• i might need to run a nix package or maven update every week probably which would amount to downloading ~200MB. that can take 10minutes; half of my travel time is wasted.
• if i use a mirror and it just takes 5 minutes to upgrade, only 1/4th of my travel time is wasted.
but generally many things are significantly slower from asia, so even if i run something on a singapore/tokyo/hong kong aws or digital ocean server, the download time differences add up to minutes compared to US servers. #2020-02-24 07:03:43 onetom if u consider that multiple ppl in the company suffer that time waste from time to time, then it's a non-negligible factor #2020-02-24 07:05:17 onetom and then i haven't even talked about ppl in china-proper, behind the GFW, who often have to go thru very slow vpns.
for them an asian mirror might be several times faster or not even being blocked, so they dont need a vpn to access... #2020-02-24 07:07:15 EmmanuelOga oh wow ok setting up that mirro rsounds like a worthy goal #2020-02-24 07:07:18 onetom because of these circumstances, im very excited to see parallel downloads in tools.deps!
now it performs just as fast as npm2/yarn/pnpm for example 🙂 #2020-02-24 07:07:47 onetom one less critique i have to listen to from former JS ppl whom im trying to teach clojure to... #2020-02-24 07:10:14 EmmanuelOga I imagine there could be a way to make it so every time you open an http connection in any program on your system it would go through your proxy #2020-02-24 07:10:36 EmmanuelOga I'm not very good at this kind of thing though so maybe it is just a stupid idea hah #2020-02-24 07:11:34 onetom proxychains doesnt do caching, but yeah, it would make sense to setup some proxies, but every ecosystem needs different kind of proxies and then some of the deps we need are private because they are our own, in-house code, or it's datomic.
setting up proxy for such an environment is not trivial at all. #2020-02-24 07:11:56 EmmanuelOga gotcha #2020-02-24 07:12:34 onetom ive experimented with various maven mirroring solution, like jfrog and artifactory, etc, but that doesn't solve brew, macos, git, nix, npm caching... #2020-02-24 07:16:56 onetom for proxychains, i've actually made a nix package:
{ stdenv, fetchFromGitHub } :
stdenv.mkDerivation rec {
pname = "proxychains";
version = "4.14";
src = fetchFromGitHub {
owner = "rofl0r";
repo = "proxychains-ng";
rev = "v${version}";
sha256 = "03wk2xpxwc7kwlq6z9jf9df1vlh6p0dn0kf826fh1k7nfaa8c4py";
};
postPatch = ''
# Temporary work-around; most likely fixed by next upstream release
#sed -i Makefile -e '/-lpthread/a LDFLAGS+=-ldl'
'';
meta = {
description = "Proxifier for SOCKS proxies";
homepage = ;
license = stdenv.lib.licenses.gpl2Plus;
platforms = stdenv.lib.platforms.all;
};
}
and using this config file for it:
[ProxyList]
socks5 127.0.0.1 1080
u can just ssh -D 1080 some.aws.basition.host to establish a tunnel, then
export MYSQL_HOST=
proxychains4 -f proxychains.conf mysql --host=$MYSQL_HOST --user=root --password -e 'show databases'
would allow accessing an RDS instance using its internal aws hostname, although the mysql cli tool doesn't support socks proxies.
just showing, to clarify what proxychains is for #2020-02-24 07:19:03 onetom also, this kind of proxying over SSH is also said to be disrupted across the GFW.
i find it a bit hard to believe, but that's what i've heard... #2020-02-24 07:21:39 onetom while all this might sound off-topic on this channel, i think it's actually not, because these kind of scenarios are the reasons, why supporting various directives in maven settings.xml or providing parallel downloads is important.
it's just ppl suffering from these issues don't really have a voice to raise them, because they don't speak good english and they are not aware where should they speak up... #2020-02-24 07:24:37 EmmanuelOga would you say clojure is popular in hong kong? #2020-02-24 07:24:52 onetom well, im personally making sure it is ;D #2020-02-24 07:26:29 EmmanuelOga nice. Good luck with that 🙂 #2020-02-24 07:26:45 onetom i've introduced clojure into almost all companies i've worked at in the past ~5years https://www.linkedin.com/in/onetom/
im not going to meetups often lately, but when i do, i also sow the seeds of clojure amongst the audience 😉 #2020-02-24 07:27:46 EmmanuelOga do you speak chinese and/or cantonese? #2020-02-24 07:28:54 onetom nope, only a bit of thai, but not at a technical level; im hungarian.
i've started to shake up the hungarian clojure community lately too on #clojure-hungary for example #2020-02-24 07:29:42 EmmanuelOga cool #2020-02-26 01:35:37 alexmiller Yes, modulo any later changes #2020-02-26 01:36:11 cfleming Ok, I’ll look at that. Seems pretty magic in there. It’s odd that that functionality isn’t part of the resolver itself, it seems to live in a Maven jar. #2020-02-26 01:36:47 alexmiller Don’t get me started #2020-02-26 08:03:04 dominicm I think I ported something I found in the maven cli code. Was a bit of a hunt. #2020-02-26 20:29:49 neupsh Hello, I have @seancorfield’s deps.edn file from https://github.com/seancorfield/dot-clojure in my ~/.clojure/deps.edn and I am running clj -A:rebel:deps to start a clojure repl with rebel-readline and tools.deps.alpha for adding libraries from within the repl. So, to be able to add library, I will have to (require '[clojure.tools.deps.alpha.repl :refer [add-lib]]) before I can call add-lib. Is there a way to auto require this in the user namespace so that when I start the repl, I have it available there? #2020-02-26 20:49:33 seancorfield @neupsh You have to split the resource/main opts:
> clj -R:rebel:deps -e "(require '[clojure.tools.deps.alpha.repl :refer [add-lib]])" -m rebel-readline.main
[Rebel readline] Type :repl/help for online help info
user=> (add-lib 'seancorfield/next.jdbc {:mvn/version "RELEASE"})
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Downloading: seancorfield/next.jdbc/maven-metadata.xml from
true #2020-02-26 20:50:55 seancorfield You could add a new alias that combines all of that -- you'll need , instead of whitespace in the -e option if you put it in deps.edn tho'. #2020-03-01 22:50:57 sam I wrote a script to aot-compile classes in my project, which I have configured to run automatically when necessary via aliases. e.g.
:test {:extra-paths ["test" "test-resources"]
:extra-deps {com.cognitect/test-runner {:git/url ""
:sha "209b64504cb3bd3b99ecfec7937b358a879f55c1"}}
:main-opts ["-i" "aot.clj"
"-m" "cognitect.test-runner"]}
I tried adding logging to my aot script using clojure.tools.logging, and found that tests started failing with java.lang.NoClassDefFoundError: clojure/tools/logging/impl/LoggerFactory
I don’t totally understand what’s going on, but I think it has something to do with the fact that the clojure compile function transitively compiles only namespaces that haven’t already been loaded. So, because I have already loaded clojure.tools.logging before calling compile, it does not get transitively compiled even though the namespaces I’m compiling depend on it.
Still, I wasn’t sure that this situation should cause a NoClassDefFoundError, since the clojure source code should still be available on my classpath, but I think it might be due to this issue: https://clojure.atlassian.net/browse/CLJ-1544. If I understand correctly, I think it’s important in my case for clojure.tools.logging to be compiled since the calling code is compiled.
My workaround was to add an explicit (compile 'clojure.tools.logging) to the script.
So my questions are:
1. Does my understanding of the problem make sense?
2. Is there an issue here with compile that should be addressed? Assuming my understanding of all this is correct, I wish compile had options about how it should handle dependency namespaces, or at least that the documentation specified this behavior (if it does, I couldn’t find it). #2020-03-01 23:09:20 seancorfield @samwagg0583 AOT has a number of serious "gotchas" and it's best to avoid it unless you absolutely need it (it's "safe" as the very last step before you build an uberjar for deployment as a complete application, but most other uses can trip you up). #2020-03-01 23:10:22 alexmiller I don’t think this is CLJ-1544 #2020-03-01 23:11:40 alexmiller Compile is transitive but it can’t necessarily “see” everything and you may need to explicitly compile things sometimes. All that is normal. #2020-03-01 23:12:50 seancorfield When code is (partially) AOT'd and then it attempts to load in (and compile on-demand) more code, you can get mismatches between the classes that the class loaders can see. Is that a fair description @alexmiller? #2020-03-01 23:14:57 alexmiller It’s more specific than that #2020-03-01 23:15:25 alexmiller Sorry, I gotta run #2020-03-01 23:13:51 alexmiller Your issue with having it already loaded is something to consider and you may need to change your compilation process to account for that #2020-03-01 23:17:17 seancorfield @samwagg0583 What is driving your need to AOT code for testing here? #2020-03-01 23:58:14 sam We are using a java library for distributed data pipelines called Apache Beam and running in Google Cloud Dataflow. Running a pipeline involves running code on a local JVM that uploads serialized java objects to remote JVMs. De-serialization on the other end requires class files, hence the AOT. #2020-03-01 23:59:31 sam My workaround of explicitly compiling classes that are already loaded seems to work okay. I just wanted to better understand the issue. #2020-03-02 00:06:08 seancorfield Ouch. Yeah, you don't have much of a choice I guess. Tho' I thought I'd read about a Beam-related library that avoided AOT recently? #2020-03-02 02:59:30 sam Thanks for this! We have looked into existing clojure libraries for Beam in the past, but I wasn’t aware of this one. I’ll definitely have to check it out further. #2020-03-02 03:33:43 seancorfield It's pretty new. I can certainly understand the authors' desire to avoid AOT for this 🙂 #2020-03-02 14:16:42 sparkofreason I've been using thurber, so far very successfully. Aaron has also been very responsive to issues and questions. It's still a bit raw, but useable, and I like the philosophy of keeping Clojure wrappers of Java APIs as thin as possible. Makes it a lot easier to figure out how to go from Java-based docs etc. to the Clojure equivalent. #2020-03-02 17:33:56 avi TIL! #2020-03-02 00:07:34 seancorfield > AOT Nothing
Fully dynamic experience. Reload namespaces at whim. thurber's dependency on Beam, Clojure, etc versions are completely dynamic/floatable. No forced AOT'd dependencies, Etc. #2020-03-02 19:20:12 fabrao hello all, using deps.edn is there any way to load dependency dynamic as boot does? #2020-03-02 19:20:35 ghadi what does that mean @fabrao? #2020-03-02 19:20:47 borkdude he might be looking for the add-lib stuff #2020-03-02 19:21:49 fabrao Humm, I think it´s what is I want, let me check #2020-03-02 19:22:21 borkdude it is in a branch, not officially out yet, but might be later #2020-03-02 19:24:33 fabrao so, for now only include new lib in :deps to include new lib and restart repl? #2020-03-02 19:28:39 seancorfield @fabrao My dot-clojure file has an alias that brings in a recent add-lib branch of tools.deps.alpha and notes on how to use it to dynamically load new libraries into your REPL without a restart. #2020-03-02 19:30:04 seancorfield I use that quite a bit to load extra libraries while I'm working. #2020-03-02 19:35:02 fabrao I have to change it to use with cider #2020-03-02 19:35:10 fabrao in emacs #2020-03-02 19:50:09 fabrao it seems that it works only in REBL enviroment #2020-03-02 19:55:59 seancorfield No, it works in a plain REPL. #2020-03-02 19:56:26 seancorfield (in fact, it specifically does not work in REBL because REBL does not use a DynamicClassLoader) #2020-03-02 19:57:22 seancorfield You don't need to change anything -- it's pure Clojure and works with any plain REPL, as long as you had the add-lib branch dependency on your classpath when you start the REPL. #2020-03-02 19:58:30 seancorfield I just start a REPL with clj -A:deps (and other options for my use). If you're using Leiningen, you'd need something that pulls in Git dependencies (I do not use Leiningen). #2020-03-02 20:04:17 alexmiller I thought we fixed the rebl thing? #2020-03-02 20:04:55 alexmiller maybe I just filed tickets to fix it generically in prepl and socket repl #2020-03-02 20:05:01 alexmiller don't remember now #2020-03-02 20:05:03 ghadi yeah tickets filed for DCL #2020-03-02 20:05:10 ghadi one ticket, I think #2020-03-02 20:05:14 alexmiller yeah #2020-03-02 20:06:10 alexmiller yeah, it's in the chute for 1.11 #2020-03-03 12:21:59 geraldodev how we can programatically get the list of loaded aliases ? #2020-03-03 12:27:50 vlaaad (.getAliases *ns*) #2020-03-03 12:38:34 vlaaad oh, sorry, this is tools-deps, I thought you are asking about namespaces #2020-03-03 12:39:26 borkdude This would be the public API for that I think (I know, this isn't about tools.deps):
user=> (require '[clojure.string :as str])
nil
user=> (ns-aliases *ns*)
{str #object[clojure.lang.Namespace 0x301d8120 "clojure.string"]} #2020-03-03 12:41:08 vlaaad ah, right, forgot about that one 😄 #2020-03-03 12:53:53 vlaaad back to original question @U0516053R :
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "2050524f56416019190d1615161412"}, :content ("[email protected]")} #2020-03-03 12:54:38 vlaaad I don’t think there is a good and reliable way to figure it out, because clj launches a new process with classpath, so no alias information is really available to the jvm #2020-03-03 12:55:57 vlaaad with that said, you can lookup your PID, lookup parent PID, run ps (linux and mac only) to find information about parent PID and extract some info from that: as you see, :out contains invocation string #2020-03-03 12:59:43 vlaaad note: ProcessHandle is available since JDK 9, previous JDKs will require some reflection and googling to get pids.. #2020-03-03 13:06:20 geraldodev @U47G49KHQ thank you #2020-03-03 13:35:03 alexmiller Can you back up to what you are trying to do? #2020-03-03 17:04:46 geraldodev I'd like to take code action based on the existence of dev alias when connecting to database. #2020-03-03 17:05:30 borkdude maybe use an environment variable for that? #2020-03-03 17:13:11 alexmiller yeah, I would add something extra that is explicit for that case, don't rely on all that process stuff above #2020-03-03 17:13:36 alexmiller system property, include dev resources in classpath, etc #2020-03-03 18:41:58 dominicm I use system props because they go into jvm-opts and they compose #2020-03-03 17:21:35 geraldodev Thank you @alexmiller and @borkdude. #2020-03-06 03:25:01 devn Is there a good guide or rationale on http://tools.deps vs other options? Several beginners have expressed confusion about lein v http://tools.deps, and lamented the fact indirectly that they’ve now needed to learn both tools while leveling up, but that almost all of their professional work has involved lein. #2020-03-06 03:27:53 devn I’m actually confused myself as a long time clojure user. Why did we need another half-finished build tool again? I think I personally understand some of the why, but there’s no doubt it’s contributed confusion to would-be clojure users. #2020-03-06 03:33:15 devn There are numerous answers here that will be acceptable to me, a long-time clojure user, but the fragmentation of beginner tutorials sucks. #2020-03-06 03:34:30 devn Sorry to be a downer. This ship has sailed, but I have to let y’all know that new users are frequently wondering why they have to learn two build tools. #2020-03-06 03:38:42 devn So, that’s kind of pointing directly at the problem. #2020-03-06 03:40:04 andy.fingerhut I can point you at resources that you may or may not find useful to point beginners at. I've got no horse in this race. #2020-03-06 03:45:32 devn I would appreciate any handy resources, and I’ll cop to soapboxing a bit here because it pains me to see beginners who are serious learners being legitimately confused by the ecosystem. #2020-03-06 03:47:54 devn I remember when I started, and people were using ivy and gradle and maven. Lein came into fashion, then cake was absorbed into lein. #2020-03-06 03:50:43 seancorfield > Why did we need another half-finished build tool again?
CLI/`deps.edn` is very specifically not a build tool. #2020-03-06 03:50:50 alexmiller ^^ #2020-03-06 03:51:03 devn you can explain this all day to new users #2020-03-06 03:51:32 seancorfield New users do not need a build tool. They need a REPL. They need to run programs. #2020-03-06 03:51:43 devn that’s nonsense imho #2020-03-06 03:52:32 devn like you can be right, but it doesn’t help new user confusion #2020-03-06 03:53:20 seancorfield The "problem" is that most books and lots of online tutorials that have been around for ages all show Leiningen as part of the first steps in using Clojure. We're in a transition period right now because Leiningen has been around for years and new tooling is appearing. This happens any time a tech community has new tooling come up to replace older, existing tooling. #2020-03-06 03:53:50 devn stop right there #2020-03-06 03:53:56 devn You said it’s not a build tool #2020-03-06 03:54:02 devn What transition? #2020-03-06 03:54:03 seancorfield There was a time when Boot was gaining ground and folks complained about that too. #2020-03-06 03:54:36 seancorfield Leiningen is a "sledgehammer": it starts a REPL, it runs programs, it bakes, it broils, it takes out the trash. #2020-03-06 03:55:18 seancorfield I see beginners tripping over problems with Leiningen all the time because of plugins they've added to ~/.lein/profiles.clj because it's "easy" rather than "simple". #2020-03-06 03:55:21 devn I expect to be on the losing end of this, but I have to really underscore the confusion for new users. I feel it’s my duty. I know that may sound hyperbolic, but this is coming from a place of love. #2020-03-06 03:55:48 devn New users do not understand this subtle distinction you’re making. #2020-03-06 03:55:56 porkostomus Not needing Leiningen to get started is a huge plus for new users #2020-03-06 03:55:58 seancorfield A lot of us here are on the front lines in #beginners all the time, helping beginners. #2020-03-06 03:56:36 devn yes, and I’ve heard directly from some of them who wonder why anyone is bringing up http://tools.deps, because all of their work is in lein world #2020-03-06 03:56:42 devn That’s exactly the confusion. #2020-03-06 03:57:27 seancorfield The official Clojure Getting Started docs lead people to the CLI and deps.edn. #2020-03-06 03:58:00 devn yes, why? The vast majority of existing production clojure projects don’t use it. #2020-03-06 03:58:07 seancorfield The latest edition of Programming Clojure uses the CLI and deps.edn I believe (right @alexmiller?) and the ClojureScript Getting Started guide uses the CLI and deps.edn. #2020-03-06 03:58:20 seancorfield Tech changes over time. There's nothing strange about that. #2020-03-06 03:58:53 seancorfield Look at all the changes in the Java world over the years... beginners now do stuff completely differently than beginners five, ten, fifteen years ago... #2020-03-06 03:59:03 devn I’ve said my piece. I don’t think it’s helpful. I think it’s confusing for new users. #2020-03-06 03:59:23 seancorfield Would you prefer tooling never changed and never improved? #2020-03-06 03:59:38 devn that’s hyperbolic #2020-03-06 03:59:54 devn I’m telling you what I know here. Anecdotal as it may be... #2020-03-06 04:00:32 seancorfield I have anecdotes too -- that many beginners find CLI and deps.edn to be much simpler to learn and use. #2020-03-06 04:02:16 devn Ok, what is depstar, and why does it exist? #2020-03-06 04:03:42 seancorfield Ask the folks who wrote it? 🙂 #2020-03-06 04:03:55 devn You maintain a fork of it #2020-03-06 04:04:05 devn no? #2020-03-06 04:04:33 devn I’m not trump to gotcha anybody #2020-03-06 04:04:50 devn But that project alone should act as sufficient evidence of the confusion #2020-03-06 04:04:50 seancorfield Yes, I found the initial version useful enough that I wanted to fix a few bugs in it, and folks seem to like my fork of it. #2020-03-06 04:05:03 devn It uberjars. #2020-03-06 04:05:12 devn It’s. A. Build. Tool. #2020-03-06 04:05:22 seancorfield Hardly. #2020-03-06 04:05:56 seancorfield It's not ant, not maven, nor gradle etc -- those are build tools. #2020-03-06 04:06:16 seancorfield Lein and Boot are build tools. #2020-03-06 04:06:51 devn again, this isn’t for me #2020-03-06 04:06:58 devn I know what you’re trying to say #2020-03-06 04:07:07 devn But for beginners this is nonsense #2020-03-06 04:07:17 devn I’ll leave it there #2020-03-06 04:10:13 alexmiller I'm confused what you would find a satisfactory answer to these queries? #2020-03-06 04:11:43 seancorfield @devn ^ #2020-03-06 04:12:30 seancorfield @alexmiller Programming Clojure uses the CLI and deps.edn in the latest edition, right? #2020-03-06 04:13:12 alexmiller yes #2020-03-06 04:14:13 seancorfield And it doesn't mention "uberjar" anywhere in the book... and only has a couple of mentions of "jar" (I just looked). #2020-03-06 04:15:09 seancorfield So a beginner can read that book end-to-end and not worry about build tools... #2020-03-06 04:15:27 seancorfield (that was what I suspected but wanted to confirm by searching the PDF) #2020-03-06 07:40:07 pyr You could look at it from the angle that there's value in having a lower level toolbox for builds. There is tremendous value as well in having a build tool with conventions (a-la lein). We can hope for a new tool or an evolution of lein, bringing back the baked in conventions which relies on tools-deps #2020-03-06 07:52:10 sogaiu check out the latest insideclojure? #2020-03-06 08:15:56 vlaaad “tools.build - shh, top secret”? 🙂 #2020-03-06 08:50:42 vlaaad A bit late to the flamewar of tools-deps vs build tools, I wanted to repeat once again how much I love clj compared to lein. It all boils down to how you run java programs. To answer the question “How do you run java programs?” one needs to have a look at java command: what arguments it accepts, what are actual, real semantics of running java programs. Actual, real semantics of running java are:
• jvm options like maximum memory (`-Xmx512`) or system properties (`-Dclojure.spec.skip-macros=true`)
• classpath: sequence of directories to look up resources (both .class files and any other type of file that might be needed for running programs) — either as a directories on file system or as archived directories (aka jar files)
• main class to run
• arguments to main class
Everything else is non-essential complexity that is imposed by a build tool.
Some of this added complexity has a lot of value: specifying dependencies to automatically build classpath, for example.
Some of it does not. For example, lets look at source path and resource path separation in leiningen. It’s accidental: you can put your source code in resource files and it will work. What good does it give you besides obscuring actual, real semantics of running java program? Lein plugins, for example, are just wrong and unnecessary abstraction: you could create a separate profile that has a dependency on a tool of your choice and run it. Automatically picked up leiningen profiles for me are still hard to understand: clj’s aliases are much more intuitive and predictable. clj command line options map much better to java command line options. Case in point: yesterday I wanted to run a lein command with system property provided from command line. I could not do it: you have to specify jvm options in :jvm-opts in project.clj, there is no way (at least that I know of) to specify them for lein run invocation.
Beginners coming from java land with equally complected tooling might have a hard time to adapt to tools-deps, but that’s because that tooling equally obscures the actual, real semantics of running java programs.
Break the cycle. Rise above. Focus on simplicity. #2020-03-06 08:56:10 dominicm https://clojurians.slack.com/archives/C6QH853H8/p1583465670122100
This message was the one I think is quite interesting. The answer is quite easy "there are 2 major competing entrypoints into clojure". I'm sure anybody can understand that. Why is there multiple battery brands that exist, etc. Because they each believe they offer something better than the other.
It's true that there's a cost to having 2 things that both provide the entrypoint. Enough people seem to think that cost pales in comparison to the benefits provided by tdeps. But those benefits are not easily rendered to someone who isn't versed enough in clojure to understand the bigger picture of elements playing out. #2020-03-06 09:18:52 teodorlu Hey @devn,
I think you're bringing some good points. I read your comments above, and I'd like to respond. I hope to be able to bring some civility.
• Personal opinion: getting started with Clojure isn't a perfect experience for new users. That you might encounter project.clj and deps.edn is evidence of that -- learning a single tool would be easier.
• Personal judgement: A common Clojure opinion is that "build tool" is a too broad category, in the sense that Gradle, Maven or NPM works. deps.edn is part of that solution. The "full solution" isn't known yet. deps.edn implementing part of that solution opens the space for other actors.
• Personal experience: A simple / pure data deps.edn is great to work with. I can download the dependencies without having to buy into how you'd like to build your project. Too much complex stuff in the project build pipeline can cause the project build pipeline to become opaque / magic.
Some questions:
1. Do you feel like the above points bring any value to the discussion you'd like?
2. Do you have a suggestion on what work should be prioritized in order to improve new user experience?
Thanks!
Teodor #2020-03-06 13:43:49 devn Oof, hearing “flame war” and “bring some civility” makes me feel pretty terrible. I talked this out with Alex for awhile last night, and Teodor, I think the second question is one I need to pose to those that have expressed their frustration to me. #2020-03-06 15:30:44 devn I sincerely apologize if I came off like a jerk. I was out of sorts last night and did a poor job of broaching the topic and keeping it constructive. facepalm #2020-03-10 11:36:27 teodorlu Yet, I think digging into frustration can be fruitful for improving the state!
I appreciate that you're spending the effort 🙂 #2020-03-06 15:50:30 seancorfield It's definitely unfortunate that Clojure didn't have something like the CLI and deps.edn from "day one" and we ended up with lein as the default : it is easy but not simple in the Clojure sense. At work we started with Leiningen because there really was no choice back in 2011. We switched to Boot in 2015, and the CLI and deps.edn at the end of 2018 I think. It's a much simpler setup, even managing a 90k line monorepo with thirty subprojects. I'm very glad that our tooling continues to evolve. Yes, it means that beginners who start from most books and tutorials today see Leiningen there but then see CLI and deps.edn in new books, new tutorials and on the official site. That's why I consider this a transition : at some point Leiningen will be the minority tooling I suspect or will at least be seen as "the old way" and the question of "which should I use?" will go away, or at least have an obvious answer. #2020-03-06 15:52:21 seancorfield The transition from monolithic Contrib to new Contrib was similarly confusing to beginners (but the community was much smaller then so the pain was less). #2020-03-06 15:53:34 seancorfield The introduction of Spec caused similar confusion I think (should I use Schema? Spec? Typed Clojure?). #2020-03-06 15:55:59 seancorfield And it's not like we don't have a bunch of options in other areas : which JSON library? Which routing library? Which web server? Learning Clojure is littered with such choices #2020-03-06 18:56:04 devn I think I may just fundamentally disagree that simple > easy for new users. I agree there are a lot of choices and that this is just the nature of the beast, but for instance, getting a basic project structure with a working test runner is a whole thing, and IMO it needn’t be this way. test-runner and clj-new exist, but they both are extra steps to simply having a basic working project structure, and then people hit the “ok how do I build an uberjar” step and now they have 3 more options to choose from that I’m aware of. I hadn’t heard about tools.build and am interested in learning more if anyone has a link handy.
lein has its warts certainly. At a certain point you definitely begin fighting with the tool. For instance, checkouts, tasks, the craziness of cljsbuild’s 9000 options, trampoline, passing java opts, and many more, but for new users you could type lein new to get a common project structure, add a dependency, write a test, type lein test, and it just worked. #2020-03-06 19:00:06 devn I know this isn’t what http://tools.deps wants to be, but I think introducing it to new users on getting started is a rougher-than-necessary entry point without a bit of ease sprinkled in. #2020-03-06 19:06:25 alexmiller I think there are multiple phases. clj is easy, probably easier than lein imo, if you want to just use a repl and write some clojure and use some deps, etc. Beyond that is "make a simple lib/app". Lein is easier in that zone (for now). And then beyond that you get to more bespoke builds, and actually I would again say clj is easier to fully customize because you are not constrained to whatever flags and switches and lifecycles lein makes available. #2020-03-06 19:07:12 alexmiller more to come in the near future for the middle zone #2020-03-06 19:08:44 alexmiller in the interim, something like sean's template (and others) can give you a slightly opinionated set of choices for making a new project based on clj. It's harder, but it's not like 10x harder, it's like 1.5x harder. #2020-03-06 19:25:57 dominicm You can make easy from simple, can't make simple from easy. #2020-03-06 19:29:33 alexmiller more importantly, simple and easy are different dimensions not a vs b. #2020-03-06 19:30:25 alexmiller so really, you can go simple+hard -> simple+easy, but you can't go complex+easy -> simple+easy. #2020-03-06 19:47:02 devn Underneath some of what I said last night, there is some personal fatigue from conversations I’ve had where people seem to use simple v easy as a way of downplaying aesthetic or ergonomic concerns by tossing those two words out without addressing the human side. New users should have it easy, full stop. To the detriment of all other things? Of course not. It’s not an either/or.
For example, I have a really difficult time imagining how having clj new and clj test would destroy the aims of the project. #2020-03-06 19:47:48 devn I’m not saying this is what should be done necessarily, but there’s no point in being pure to the detriment of new users. #2020-03-06 19:58:28 alexmiller I certainly agree that simple does not mean hard #2020-03-06 19:59:56 alexmiller clj new and clj test seem like making clj into something that it's not though (a build tool). it's a tool for running programs. template gen and test runners are programs. #2020-03-06 20:00:36 devn totally agree #2020-03-06 20:00:45 alexmiller having ways to discover useful programs to run or templatize/kickstart how to do so seem like good things #2020-03-06 20:01:25 devn but like, should I just make a thing called dmk that does all of the clj stuff and adds two commands for new users? #2020-03-06 20:01:37 devn that seems bizarre to me? #2020-03-06 20:02:08 alexmiller well I wouldn't do that :) #2020-03-06 20:02:37 devn haha, I’m asking then honestly, where would something like that go? It’s never in the cards? #2020-03-06 20:03:03 alexmiller I don't have a pat answer to that - finding one is a design process #2020-03-06 20:03:29 alexmiller these things literally exist now #2020-03-06 20:03:51 alexmiller as programs that you can run now with clj #2020-03-06 20:04:10 alexmiller and as templates that you can create/install #2020-03-06 20:05:21 alexmiller is the problem that we have not done a good job explaining that or that that is not easy enough. and if the latter, what would make it so? #2020-03-06 20:08:12 alexmiller if you added a bit of stuff to your ~/.clojure/deps.edn, then it's literally clj -A:test or clj -A:new etc which seems pretty close to what you have above #2020-03-06 20:09:33 devn It kind of reminds me of clj-rewrite and the evolution of “just update my deps” or “just add the damn test runner” #2020-03-06 20:10:56 devn Brb #2020-03-06 20:19:50 mike_ananev in templates I use make utility, so it is very easy to perform different tasks #2020-03-06 20:34:58 seancorfield If someone does add the :new alias to create a new app or lib project, the generated project includes the :test, :jar/`:uberjar`, :install/`:deploy` aliases so it all becomes as "easy" as lein for basic usage. So the missing piece is really only that initial project creation piece I think. #2020-03-06 20:36:21 seancorfield Strangely, when we switched from lein to boot at work, "new" wasn't a thing for Boot either and I ended up forking lein-new (which wasn't a thing for Leiningen in the very early days, by the way!) into boot-new and it was quite some time before that was integrated into the core tool. #2020-03-06 20:36:25 devn Heh, I mean, here we are. http://tools.deps is not a build tool but there are 3 choices for making an uberjar. It’s too simple to add any porcelain commands so there are at least three things that add that type of functionality. That’s my gripe I guess. #2020-03-06 20:36:44 seancorfield So there's a history of tooling not including an "easy" way to start a new project from scratch 🙂 #2020-03-06 20:37:15 devn Oh yeah! I actually worked on the boot lein template thing I think... #2020-03-06 20:37:17 seancorfield > 3 choices for making an uberjar
Oh, c'mon! There are far more than just three choices! :rolling_on_the_floor_laughing: #2020-03-06 20:37:18 devn Long time ago #2020-03-06 20:37:29 devn haha #2020-03-06 20:38:49 seancorfield I think far more beginners are stumped by the "how do I build a simple web app" question -- given the plethora of choices they get offered at that point. And heaven help them if someone says lein new luminus myapp and then they have to figure out what that produces! #2020-03-07 10:27:08 practicalli I can only share my experiences, but build tools and running a repl are fairly low on the confusion list for those new to coding with Clojure. At ClojureBridge London we have introduced over 300 students who are either new to coding or on their first language. We had been using Leiningen for a long time and now use Clojure CLI and Leiningen,
Clojure CLI is far quicker to install and run, deps.edn has less in it so is easier to explain. As there are numerous plugins (aliases), it's much easier to gradually introduce new features into how Clojure is used. We only really use clj-new and figwheel-main You can also have both Leiningen and deps.edn configuration in a project, so there is a very simple migration (especially if there are relatively few dependencies).
If you have invested a lot of time learning Leiningen or rely heavily on Leiningen plugins that do some magic, like lein-ring, then switching to deps.edn configuration does require a bit of effort, but you don't have to switch. #2020-03-08 05:09:00 porkostomus Sounds like a great community! I often feel like an oddball for (besides all the usual reasons) choosing to dive into Clojure without significant experience in another language. It has been an interesting ride, and I would love to talk to others who have similar experiences. #2020-03-10 11:49:44 teodorlu I'd like to add some comments to the discussion on deps.edn's ease of use:
• To me, deps.edn seems like the right platform to make build tools on. It standardizes a part that can and should be standardized.
• How to build a user-friendly product on top of that can then be tacled as the next problem. It would make sense to converge on standard build tooling, but that, to me, seems like the next problem "up the chain".
The Clojure way has seemed to me like "doing the right thing, and letting it take the time it requires". I'm eager to see what build tooling we can have on top of deps.edn when some time has passed. #2020-03-10 11:51:49 teodorlu Relevant dichotomy: bottom up/ top down. Bottom-up: small functions, composition. Platform first. Common with Clojurians. Top-down: focus foremost on the end-user experience, and try to improve that incrementally. Common in JS-land.
To me, bottom-up rhymes with simple-first, whereas top-down rhymes with easy-first. #2020-03-10 11:51:49 teodorlu Relevant dichotomy: bottom up/ top down. Bottom-up: small functions, composition. Platform first. Common with Clojurians. Top-down: focus foremost on the end-user experience, and try to improve that incrementally. Common in JS-land.
To me, bottom-up rhymes with simple-first, whereas top-down rhymes with easy-first. #2020-03-10 17:59:04 rickmoynihan Interesting observations. I guess neither top-down nor bottom up guarantee either simplicity or ease in the final result; but yeah I think what you’re saying generally holds. And you can also do both together and meet in the middle. As a heuristic though if you wanted simplicity you’d tend to start bottom up, and if you wanted ease you might prefer top-down — though they’re not necessarily mutually exclusive. #2020-03-11 09:09:31 slipset One of the things that I find interesting in this perspective is that Clojure has slurp/spit whereas I’ve never figured out how to read a file in Java (prior to Java7)
Here it seems like Clojure has taken the top down approach, whereas Java took the bottom up approach with the streams and readers and what not. #2020-03-11 09:21:00 rickmoynihan Nah, clojure also has the simple approach too with #2020-03-11 09:55:07 teodorlu @U06HHF230 I agree that this shouldn't be taken quite literally. End user experience definitely is important! And we have to keep that in mind. I'm quite fond of how the JS-ecosystem in general caters to an ease-in of new users. #2020-03-11 09:56:55 teodorlu @U04V5VAUN Nice catch. I agree that the "easy" Clojure approach complects resource usage -- there's no way to decouple streaming out of spit/slurp.
Abstraction layers is relevant here, though. In the context of "slurp reads an external resource into pure data (string)", I think it makes sense. #2020-03-11 09:58:24 slipset It totally makes sense, and, Clojure lets you compose other “abstractions” when it comes to reading stuff from a file. Thing is that slurp makes the common case easy. #2020-03-11 20:09:41 unbalanced no sure if this is a tool-deps specific question exactly, does anyone have any suggestions on how to troubleshoot this?
$ clj -R:nREPL figwheel.main -bo min
Cloning:
#2020-03-11 20:10:37 unbalanced I am able to clone it directly with git #2020-03-11 20:10:50 alexmiller what os? #2020-03-11 20:10:56 unbalanced red hat >.< #2020-03-11 20:11:12 unbalanced $ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL=""
BUG_REPORT_URL=""
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
#2020-03-11 20:11:15 alexmiller it uses the ssh agent to obtain identities #2020-03-11 20:11:20 alexmiller so - is that running? #2020-03-11 20:11:35 alexmiller ssh-agent #2020-03-11 20:11:48 alexmiller if so, does it have your identity ssh-add -l #2020-03-11 20:11:59 alexmiller if not, add it #2020-03-11 20:12:14 unbalanced checking #2020-03-11 20:12:26 alexmiller those cover 90% of these issues #2020-03-11 20:14:24 unbalanced appreciated. I turned it on and got a new error. Trying to figure out how to add my key now. #2020-03-11 20:15:14 alexmiller ssh-add ~/.ssh/id_rsa
#2020-03-11 20:15:58 unbalanced $ clj -R:nREPL figwheel.main -bo min
Cloning:
#2020-03-11 20:16:03 unbalanced does that seem like a different error? #2020-03-11 20:16:22 alexmiller yeah, that seems bad #2020-03-11 20:16:24 unbalanced (after following steps of turning on ssh-agent and adding key) #2020-03-11 20:16:29 unbalanced hahahaha #2020-03-11 20:16:32 alexmiller might be in how you're running ssh-agent #2020-03-11 20:16:32 unbalanced I believe you #2020-03-11 20:16:49 alexmiller you've walked off my knowledge in this area :) #2020-03-11 20:17:13 unbalanced alright! good start. much appreciated #2020-03-11 20:19:44 alexmiller sudo mount /tmp -o remount,exec internet suggests maybe /tmp needs exec ? #2020-03-11 20:20:46 unbalanced hmm #2020-03-11 20:21:47 alexmiller or set a different tmp dir #2020-03-11 20:22:32 alexmiller clj -.tmpdir=/my/tmp -R:nREPL figwheel.main -bo min #2020-03-11 20:23:07 alexmiller the git ssh stuff happens through jsch which talks to ssh-agent via jna. it appears jna will make and use stuff in the tmp dir #2020-03-11 20:24:17 unbalanced interesting #2020-03-11 20:24:27 alexmiller I've never seen this before, so you get a cookie #2020-03-11 20:24:37 alexmiller 🍪 #2020-03-11 20:24:38 unbalanced hahahahahaha #2020-03-11 20:24:41 unbalanced yesssss #2020-03-11 20:26:21 unbalanced almost done setting all this up jeeze #2020-03-11 20:27:46 unbalanced now I'm getting UnknownHost key, but at least we're making progress in some direction #2020-03-11 20:28:12 unbalanced of course I see there are 6 ssh-agents running now 😑 #2020-03-11 20:35:45 unbalanced that fixed it #2020-03-11 20:35:58 alexmiller 🍪 #2020-03-11 20:36:13 unbalanced 😮 holy snap #2020-03-11 20:37:46 unbalanced that's goin on the fridge #2020-03-11 20:37:55 unbalanced okay last problem, most of the things downloaded #2020-03-11 20:38:06 unbalanced Error building classpath. Failed to read artifact descriptor for org.clojure:clojure:jar:1.10.1
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:clojure:jar:1.10.1
#2020-03-11 20:38:42 unbalanced @alexmiller is there a way to comment on that issue? #2020-03-11 20:39:05 unbalanced or just... include that link in there I guess #2020-03-11 20:39:29 unbalanced specifically this is what I did:
$ ssh-keyscan -H -t rsa >> known_hosts
#2020-03-11 20:42:53 alexmiller done #2020-03-11 20:50:24 unbalanced last piece of the puzzle was a stale .m2 repo that had wrong perms #2020-03-11 20:51:19 unbalanced do you know where the git stuff gets cloned to? I don't see it in .m2 #2020-03-11 20:56:04 unbalanced .gitlibs #2020-03-11 20:56:38 unbalanced sometimes it helps if I just post the questions and then get embarrassed and try harder #2020-03-12 16:04:28 danielcompton I'm wanting to write a small Java file and use it in my Clojure app using deps.edn and clj . Is there anything similar to :java-source-paths or another way I can accomplish the same goal? My other option is to write this and create a JAR for it and deploy it somewhere or keep it as a local resource #2020-03-12 16:05:41 dominicm No equivalent. No compilation done by tdeps. #2020-03-12 16:06:18 dominicm You can use javac and have it output to target, and run that shell script before starting clj #2020-03-12 16:18:35 danielcompton Might be easier to rewrite the algorithm to Clojure 🙂 #2020-03-12 16:39:02 alexmiller there are some tools.deps add-ons that can compile as well, like badigeon #2020-03-12 16:48:20 mike_ananev it's a wrapper for badigeon #2020-03-12 16:49:01 borkdude maybe a silly question, but why does badigeon need a wrapper? it's already a tool? #2020-03-12 16:50:20 mike_ananev @borkdude no, badigeon is not a ready tool. it is low level api for managing project distributed as sources, which need to be adapted. #2020-03-12 16:51:29 borkdude ah ok, I see now #2020-03-12 17:52:27 dominicm Javac is pretty good though #2020-03-12 20:48:39 alexmiller @dominicm if you are suggesting that libraries should maintain a directory full of compiled classes, I think that is a bad idea and you should not do that #2020-03-12 20:51:26 alexmiller if you want to distribute compiled artifacts, you can already do that by putting them in a jar. we don't typically do that in Clojure because the source is so much more flexible. #2020-03-12 20:53:29 dominicm I'm not. I'm saying all of the infrastructure around this, code laying around to do the compilation on startup. #2020-03-12 20:53:59 imre @alexmiller if you call (compile 'foo) twice, will the second be almost a no-op as it sees it's already compiled? #2020-03-12 20:57:06 alexmiller yes #2020-03-12 20:58:26 alexmiller you can try it in that project I posted if you like #2020-03-12 21:03:18 imre cheers #2020-03-13 16:35:19 unbalanced I'm back with more git issues #2020-03-13 16:36:37 unbalanced Cloning:
#2020-03-13 16:36:47 unbalanced trying to help a coworker get it running #2020-03-13 16:36:50 unbalanced this is on mac #2020-03-13 16:37:01 unbalanced so far we've done the following #2020-03-13 16:37:49 unbalanced if [ -z "$SSH_AUTH_SOCK" ] ; then
eval `ssh-agent -s`
ssh-add
ssh-add -k ~/.ssh/id_rsa # add keys -- did this for all private keys
# add github keys
ssh-keyscan -H -t rsa >> ~/.ssh/known_hosts
fi
#2020-03-13 16:39:13 unbalanced git clone works #2020-03-13 16:40:16 ghadi do they also have ~/.ssh/config entries? @goomba #2020-03-13 16:41:49 unbalanced I haven't checked -- is that important? #2020-03-13 16:42:08 unbalanced there is no ssh config #2020-03-13 16:42:45 unbalanced #2020-03-13 16:43:42 unbalanced wow #2020-03-13 16:43:47 unbalanced renamed stuff and it worked #2020-03-13 16:44:19 unbalanced how on earth did you know that #2020-03-13 16:47:00 alexmiller what error are you getting? #2020-03-13 16:47:29 unbalanced org.eclipse.jgit.api.errors.TransportException: #2020-03-13 16:47:54 unbalanced it was tied to the .ssh/config somehow #2020-03-13 16:51:02 unbalanced (it's working now, thank you @ghadi) #2020-03-13 16:51:17 ghadi JSch, the underlying library used, tries to read the .ssh/config for some things instead of consulting the running ssh-agent #2020-03-13 16:53:58 unbalanced I don't suppose :git/url supports tags/branches, and/or would a PR to that effect be welcome? #2020-03-13 16:54:45 unbalanced I'm using this extensively and I have 11 (and growing) repos (we're using enterprise git so I can't publish to clojars) and being able to tags/branches would be nice #2020-03-13 16:55:32 alexmiller so as above tools.gitlibs does support tags or branches #2020-03-13 16:55:50 alexmiller clj intentionally supports only full shas #2020-03-13 16:56:09 unbalanced because tags/branches are mutable ? #2020-03-13 16:56:19 alexmiller yes, it totally breaks classpath caching #2020-03-13 16:56:36 unbalanced tags too? #2020-03-13 16:56:44 alexmiller same #2020-03-13 17:13:26 dominicm Well, tags are meant to be immutable. You have to force push them. #2020-03-13 17:27:17 alexmiller you see how somebody else could break your build with that though, right? #2020-03-13 17:45:40 dominicm Oh yeah. I wasn't sure how defensive we were being really. I understand that one changes future builds, and one is cryptographically difficult to change future builds. If you want cryptographic protection then it's okay. #2020-03-13 18:06:34 dominicm Maven central do that through policy, but there's no requirement that happens though (I think) #2020-03-13 16:57:00 unbalanced gotcha... bummer #2020-03-13 16:57:40 unbalanced I guess including a "force recalculate" directive would be out of the question? #2020-03-13 16:57:59 alexmiller that does exist (-Sforce) but you would need to basically do it on every clj call #2020-03-13 16:58:56 alexmiller I would not rule out some way to support this use case, but would want to start from the big picture #2020-03-13 16:59:01 unbalanced gotcha. I'm just trying to make the workflow for this work in a less error-prone way. #2020-03-13 16:59:38 alexmiller the other similar but different thing people have been poking at is monorepo with embedded local deps #2020-03-13 17:00:47 unbalanced I'm gonna DM you the details that I can't put here on what the situation is #2020-03-13 17:01:18 alexmiller another you might want to consider is using a private shared repo (either something like nexus or even a private s3 repo) #2020-03-13 17:02:04 unbalanced so for each of those repos you see ... every time I make a commit, I need to update the shas. And then I need to update the READMEs for each one. And god forbid they have dependencies on one another --- need to do all that by hand #2020-03-13 17:02:19 alexmiller I get it - I've used a private shared repo in this situation #2020-03-13 17:02:36 unbalanced is that like a "private clojars"? #2020-03-13 17:02:42 alexmiller yeah #2020-03-13 17:02:51 unbalanced ok -- that would be a much better solution, yes #2020-03-13 17:02:53 alexmiller maven repos can be just a file system #2020-03-13 17:03:02 alexmiller and private s3 repos are exactly that #2020-03-13 17:03:21 unbalanced I did not know that. How do you specify an alternate .m2? #2020-03-13 17:03:42 alexmiller nexus is actually an app and can do a lot more for you (like mirroring all your external deps in case maven central is unavailable or something) #2020-03-13 17:03:51 alexmiller you don't need an alternate .m2 #2020-03-13 17:04:02 alexmiller for this #2020-03-13 17:04:15 alexmiller if you're asking separately, you can use a :mvn/local-repo key in your deps.edn #2020-03-13 17:04:27 unbalanced innnteresting #2020-03-13 17:04:53 unbalanced okay, thank you, I'll look into that. 🙏 #2020-03-13 17:08:59 unbalanced which category on tools would that be under? #2020-03-13 17:09:13 unbalanced versioning and releasing? #2020-03-13 17:10:20 alexmiller which part? #2020-03-13 17:10:40 alexmiller making an artifact -> packaging #2020-03-13 17:11:10 alexmiller but I guess also some of versioning and releasing #2020-03-13 17:11:31 unbalanced no I meant a tool for where to host #2020-03-13 17:11:37 unbalanced the files #2020-03-13 17:11:44 unbalanced i.e. nexus or similar #2020-03-13 17:12:25 alexmiller oh, I don't think you'll find that theree #2020-03-13 17:12:53 unbalanced just making sure I'm not crazy #2020-03-13 17:12:57 alexmiller but you could look at any maven repository docs. https://www.deps.co/ is built by Daniel Compton from the Clojure community #2020-03-13 17:13:04 unbalanced 😛 well I am technically a little @goomba #2020-03-13 17:13:54 alexmiller those are common ones #2020-03-13 17:14:43 alexmiller I think there's a thing built into git too? #2020-03-13 17:15:04 unbalanced WHAAAT #2020-03-13 17:15:17 unbalanced yay that's a tool I'm actually allowed to use! #2020-03-13 17:15:38 alexmiller just googling, not sure that particular article is the platonic ideal #2020-03-13 17:17:25 alexmiller there's some tool for this too that I can't remember the name of #2020-03-13 17:40:46 dharrigan I use Nexus OSS #2020-03-13 17:40:48 dharrigan works well #2020-03-13 18:01:39 alexmiller ditto #2020-03-13 19:32:33 borkdude maybe he can try -Sforce? #2020-03-13 20:00:10 hiredman they should delete .gitlibs/libs/helix #2020-03-13 20:12:49 hiredman or maybe .gitlibs/libs/Lokeh/helix #2020-03-15 15:23:10 teodorlu I'd like to comment on the previous discussion of referencing with tags.
• Personal opinion: a utility to rewrite a git ref (HEAD, some-tag, some SHA letters) into a full-blown 40-char SHA might help ease of use. That may be a task for a more high-level tool, though; it seems like more of a UI concern. Perhaps an alias.
• Personal opinions: refs should be resolved immediately, and only SHAs make it into the data
clj -A:install-dep --git-url --git-ref HEAD
I'm guessing this might not be novel ideas, as I feel that tools.deps already embodies this kind of thinking. #2020-03-15 16:24:33 alexmiller Have you looked at -Sresolve-tags option #2020-03-15 16:25:10 alexmiller It resolves tags to shas in your deps.edn #2020-03-16 18:43:00 teodorlu You're way ahead of me 💯 #2020-03-15 23:19:59 Jag Gunawardana I want to include my database migrations in my Uberjar. I am using juxt edge with onejar to build. In lein resource-paths did this. I am using ragtime with edn files. The only way I can get this to work is using “jar uf...” to update the jar after building it. There must be some deps .edn magic? #2020-03-15 23:41:38 seancorfield @jag I've no idea how Juxt/OneJar handle that but with depstar you would just use an alias that adds the resources (via :extra-paths) when you ask it to build the JAR. #2020-03-15 23:42:43 seancorfield We build uberjars at work using depstar that include DB migrations, HTML templates, and all sorts of non-code stuff. #2020-03-16 08:20:31 dominicm @jag they should be included by default unless they're not in a default alias. If they aren't you can put "-A:alias" to pack. #2020-03-16 15:59:44 Jag Gunawardana Thanks for the suggestions, looks like moving to depstar seems to be the best long term solution. I like the approach taken in juxt edge project, but the docs around build are a bit too concise. #2020-03-16 16:40:17 dominicm Sorry about that, is there somewhere I can improve them? #2020-03-18 08:53:54 Jag Gunawardana I think a few more use cases for some of the tools, or general cases would work e.g. “How to include resources in the JAR file”. I ended up using Sean’s fork of depstar with edge.
clojure -A:prod:depstar -m hf.depstar.uberjar myjar.jar
and running
java -cp myjar.jar clojure.main -m edge.main
That seems to work fine.
I can add something to the docs on this if it helps? #2020-03-18 10:29:05 dominicm I'd love to take a look at a pr if you have an idea for this :) #2020-03-19 15:37:05 Eamonn Sullivan Hi, probably a quick question: Can I use a private github repo as a dependency in deps.edn? I'm trying, but getting Authentication is required but no CredentialsProvider has been registered. If it is supported, how do I provide my credentials (a github access token, I guess)? #2020-03-19 15:39:36 ghadi you can, but through ssh:// + an ssh-agent, not via the https:// url #2020-03-19 15:39:36 ghadi you can, but through ssh:// + an ssh-agent, not via the https:// url #2020-03-19 15:50:28 Eamonn Sullivan Sorry, I thought I hadn't run ssh-agent and it was working, but I was actually in the wrong directory. 😞 #2020-03-19 15:53:57 ghadi please describe exactly what you did, and what the result was #2020-03-19 15:57:46 Eamonn Sullivan I added to the dependencies. My private repo is the last one listed below. I then try to run clj and the result is Error building classpath. #2020-03-19 16:01:07 ghadi what is the output of ssh-add -l #2020-03-19 16:01:17 ghadi and is your key an ed25519 key? #2020-03-19 16:02:06 Eamonn Sullivan here it is. #2020-03-19 16:03:57 Eamonn Sullivan And, no, it's not an ed25519 key, as far as I can tell (they both say RSA). #2020-03-19 16:06:55 ghadi if you have a bunch of entries like defaults in $HOME/.ssh/config consider moving that file out of the way and retrying #2020-03-19 16:07:05 ghadi temporarily #2020-03-19 16:07:27 ghadi the ssh library prioritizes that file even when there is an ssh-agent present #2020-03-19 16:09:38 ghadi (unfortunately) #2020-03-19 16:09:42 Eamonn Sullivan Ah, that worked. I need those entries in config for various things, so I wonder what I need to do to not confuse things. #2020-03-19 16:12:30 Eamonn Sullivan But, anyway, I can now build, so that's a big improvement. Thank you! #2020-03-19 16:13:19 ghadi my pleasure #2020-03-19 21:39:04 sam I’m getting an unexpected transitive dependency resolution in my deps.edn project. This is what my dependency tree looks like:
my-project -> direct-dep -> trans-dep1 (version 1)
\
-------> trans-dep2 -> trans-dep1 (version 2)
• direct-dep is a git dependency
• direct-dep has a pom.xml file that serves as the manifest
The problem is that trans-dep1 (version 2) is on the classpath instead of version 1 #2020-03-19 21:41:26 sam Am I correct in thinking that version 1 of trans-dep1 should be the expected resolution (assuming I’m describing this correctly and am not doing something dumb I’ve overlooked) #2020-03-19 21:43:09 ghadi tools.deps prefers latest versions first #2020-03-19 21:52:30 sam Oh thanks, I totally missed that. #2020-03-19 21:53:46 sam Is there a rationale for that written somewhere? #2020-03-19 21:57:35 sam Awesome, thanks. I appreciate it. #2020-03-19 21:58:07 alexmiller I don't know if I have a good written thing to point at #2020-03-19 22:06:56 sam Np, I think I’ve seen the spec-ulation talk. I think I vaguely remember this. In any case, I can imagine an argument that chosing the latest version is simpler, easier-to-understand behavior compared to the behavior in lein/mvn that I was expecting. #2020-03-19 22:25:40 seancorfield The latest episode of the ClojureScript Podcast is very good https://clojurescriptpodcast.com/ -- Alex talks about tools.deps and covers this specific issue... #2020-03-19 22:30:12 alexmiller oh yeah, that too. we recorded it months ago so it's hard to remember that. :) #2020-03-19 22:40:39 seancorfield It's a really good all-round discussion about tools.deps and the new CLI stuff. Definitely worth pointing folks at for any Qs regarding the "philosophy" of it (including the SemVer stuff). #2020-03-20 15:38:43 grzm I have a project which has a deps.edn dependency to a private https://github.com repo, Clojure deps is failing to pull the dependency with:
Cloning:
Seems like it’s an issue with my SSH keys, but I’m a bit puzzled as I can clone this same dependency from https://github.com from the command line with git clone without any problem, and the necessary keys show up when using ssh-add -l. Hints on what I might be doing wrong? #2020-03-21 09:16:21 practicalli the #clj-on-windows channel can also provide help with this #2020-03-22 01:22:49 colinkahn Oh nice, thanks! #2020-03-20 23:57:55 sogaiu it definitely has relevant info that people have referenced here from time to time.
the scoop option mentioned at the bottom of the page appears to be somewhat popular. would recommend checking that out. #2020-03-20 23:58:16 sogaiu i have been using that successfully. #2020-03-21 23:29:27 practicalli How would I include the Java sources in the :path of my deps.edn configuration?
The sources are currently in a src.zip file, from the install. Is it as simple as using "src.zip" to the :path configuration? Or would I need to extract and a the specific directories relative to the Clojure project?
Also wondering how to include these sources to multiple projects, either using an alias in ~/.clojure/deps.edn or creating a symbolic link to the src.zip file (or extracted directory). #2020-03-21 23:49:16 andy.fingerhut Neither the clj nor clojure commands will do anything with Java sources, unless you somehow use them to run a program that looks for and does something with Java source files. #2020-03-22 01:07:06 potetm OR you can compile the src w/ javac into target/classes and add "target/classes" to your :paths #2020-03-22 07:34:27 dominicm I think the src.zip is referring to the ones that come with the jdk. They're used by cider to allow you to jump to source on java code. #2020-03-22 07:35:27 dominicm @U05254DQM I think zip files are fine on the classpath. A :local/root should be fine. #2020-03-22 08:41:29 practicalli @U09LZR36F that sounds what I am looking for, thank you. #2020-03-22 11:15:56 practicalli It works. I have a deps.edn as follows
{:path
["src" "resources"]
:deps
{org.clojure/clojure {:mvn/version "1.10.1"}
;; Java 11 sources
java-sources {:local/root "/usr/lib/jvm/openjdk-11/lib/src.zip"}}}
I can now browse java sources in Spacemacs with clj-find-var . Thanks everyone. #2020-03-22 13:09:31 dominicm Remember that the location will vary for everyone :) #2020-03-22 11:19:36 pithyless Trying to debug this kind of warning:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:~/.m2/repository/org/slf4j/slf4j-nop/1.7.25/slf4j-nop-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:~/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
When I run clojure -Stree I do not see any mention of slf4j-nop. Any ideas on how I can find out what is pulling in this dependency (so I can add an :exclusion)? #2020-03-22 11:26:09 pithyless ^ Nevermind, PEBCAK. #2020-03-24 14:07:25 holyjak Hi! I have added :mvn/repos {"my-company" {:url "...."}} but I want deps to also resolve dependencies from my maven local repo. How do I add it, preferably w/o hardcoding the path of my home directory? #2020-03-24 14:08:25 alexmiller it will already do that #2020-03-24 14:08:53 alexmiller assuming you mean ~/.m2/repository - that's actually not a repo, but a local cache of external repos #2020-03-24 14:10:04 alexmiller as a cache, it's checked first though so you can install stuff locally that you never deploy and it will be used #2020-03-24 14:15:38 holyjak thanks, that is exactly my use case. I found out that the jar was wrongly published to the local cache. #2020-03-24 23:29:17 athomasoriginal Terminology is always as touchy topic, but would it be fair to refer as tools.deps.alpha as a package manager? If not, how would you describe the distinction?
The goal for myself to group clojure tools into buckets that fit my mental model. I am happy to update my model, but I just want to see if my current understanding fits 🙂 #2020-03-24 23:36:19 andy.fingerhut I do not know of any precise definition of the term "package manager" that everyone would agree with, across different programming languages, but it seems to me that tools that are called by that name often do more things than tools.deps.alpha does. #2020-03-24 23:36:58 andy.fingerhut This is too many words, but the http://clojure.org page introducing tools.deps.alpha describes it thus: "a library providing a functional API for resolving dependency graphs and building classpaths that can utilize both Maven and other providers of code or artifacts" #2020-03-24 23:37:22 andy.fingerhut Perhaps calling it a "dependency resolver" briefly captures the essentials? #2020-03-24 23:40:49 athomasoriginal My thoughts exactly, Andy. I agree that it would be a stretch to compare it as this would lead to further comparisons to other tools like npm #2020-03-25 01:44:18 alexmiller tools.deps is fundamentally about making classpaths, and incidentally about transitive dependency resolution #2020-03-25 01:45:03 alexmiller I would not call it a package manager per se #2020-03-25 01:45:21 alexmiller clj is about running programs #2020-03-25 10:21:23 rickmoynihan I’ve been meaning to speak about this issue for a while; not sure if others have run into it or not.
I have a private library in a private github repo; that we’re requiring via tools.deps.
Every so often, in circumstances I don’t fully understand, but it usually appears to be when I do things like change branches (and I suspect the sha in that dep has changed) I get the following exception: #2020-03-25 10:21:31 rickmoynihan #2020-03-25 10:25:13 rickmoynihan When it occurs I usually spot that my ssh-agent is somehow no longer running; so restart that but then get: #2020-03-25 10:25:52 rickmoynihan #2020-03-25 10:27:13 rickmoynihan Then clearing the lock file fixes it #2020-03-25 12:19:27 alexmiller We’ve had people report this but I’ve never been able to repro it #2020-03-25 14:00:20 rickmoynihan It happens to me quite often #2020-03-25 14:05:41 rickmoynihan there are a few other things that might be relevant:
1. some-private-lib is actually included in our deps.edn multiple times; with different :deps/root s
2. In dev I start a bunch of services with overmind a Procfile process manager. It boots them in parallel. One of those services is my clj repl; another is shadow-cljs that references the same deps.edn; so when resolving deps two processes are occaisionally concurrently resolving/fetching deps in parallel. #2020-03-25 14:07:52 rickmoynihan I suspect 2 might be problematic… so sometimes try and resolve the deps first; then run overmind… though I do at times forget — it usually seems to work without any apparent problems, though it does seem a bit dangerous on my part. #2020-03-25 14:09:47 Eamonn Sullivan I saw the first error just the other day. I had two small projects, one using a github ref using one hash (that checked out the repo fine) and then moved to another with a newer hash (but the same repo) and saw the above error when trying to build. I went into the directory and deleted the repo and tried again and it worked. Is that supposed to work? #2020-03-25 14:11:16 rickmoynihan I think I’ve seen it a similar scenario to that too #2020-03-25 14:11:38 rickmoynihan but this one is definitely within a single project — though changing branches; with a changing sha on the dep which probably amounts to the same thing #2020-03-25 14:16:54 alexmiller with the newer clj's, resolution happens in parallel, so even in a single project, you could encounter that #2020-03-25 14:19:32 alexmiller you can turn that off with -Sthreads 1 #2020-03-25 14:42:11 rickmoynihan Ok but this is in a single project; but with two concurrent processes potentially resolving the same deps with tools.deps. Is that safe if the processes aren’t coordinated? #2020-03-25 15:12:45 alexmiller don't know #2020-03-25 12:20:21 alexmiller The first error that is #2020-03-25 13:36:49 WhittlesJr Hi,
I love the ability of tools-deps to target git repos at a certain hash. I've operated under the assumption that the hostname of the repo will stay the same forever for private repo dependencies. But that's proven not always to be the case.
If a repo moves to a new hostname, it breaks every historical project state that referenced it.
Yocto has a feature called a "repo manifest" that allows the repository configuration to be a lot more fluid, and effectively solves this problem. Does tools-deps have an equivalent option? #2020-03-25 13:38:59 alexmiller no #2020-03-25 13:39:09 alexmiller what is the repo manifest? #2020-03-25 13:39:21 alexmiller got a link or something? #2020-03-25 13:49:38 WhittlesJr I suppose there are workarounds for the issue at the OS-level, but this would be a pretty slick feature. #2020-03-25 14:05:03 dominicm @alex.joseph.whitt is this like having a proxy or alternative source for a repo? I'm struggling to understand. #2020-03-25 14:16:22 WhittlesJr I'm not a Yocto user myself, but I believe the idea is that you can have a locally-defined manifest of repo sources. For each dependency, Yocto will look through them all in order and try to resolve the dependency in each.
That solves the problem of trying to build an old version of your project after your git server hostname changes. The problem is really only sticky when you have multiple internal Clojure dependencies that all point to the same hostname. You have to go through each dependency, branch off of the contemporary commit, update the hostname, commit, then update the hashes on projects that pull in that dependency. Kind of hellish unless you redirect the old hostname at the DNS level or something. #2020-03-25 14:21:37 alexmiller I mean, you are defining a lib and a coordinate (with git config) in your deps.edn. You can change that coordinate/git config but keep the lib name to point it elsewhere. #2020-03-25 14:26:31 alexmiller I guess there could be a layer of indirection added in front of the git url #2020-03-25 14:26:41 alexmiller needs a lot more thought #2020-03-25 14:51:04 dominicm Another use case for this: the bank I work at has a github mirror which is both faster and allows clones from machines that otherwise can't.
This would be useful for us. #2020-03-26 10:31:28 daveliepmann Hi folks, I've got a head-scratcher with transitive dependencies. I found a workaround but it doesn't seem right.
We have a new library which depends on a Java lib living in the BeDataDriven public repository: http://docs.renjin.org/en/latest/library/project-setup.html
We wish to consume this library in a separate project. However, we get an error when starting a REPL via clojure CLI (or CIDER):
Error building classpath. Could not find artifact org.renjin:renjin-script-engine:jar:3.5-beta43 in central ()
This error goes away if I include the BeDataDriven repo to the consumer's :mvn/repos. Like I said, this is a fix but it doesn't feel correct. Is this by design?
I've created a minimal reproduction (three tiny files) of the issue in this repo: https://github.com/daveliepmann/minimal-transitive-deps-example The workaround is commented out. I get the error above when running clojure from the command line in min-consumer directory. I am running the latest clojure from brew.
The actual lib is at https://github.com/appliedsciencestudio/rdata/ and the consumer deps.edn fails with both git and local deps:
clojure
appliedsciencestudio/rdata ;; {:local/root "/Users/daveliepmann/src/forks/rdata"}
{:git/url ""
:sha "6dff85aa7b83a9ce85eb8d70b0672a3adf0dc7f2"}
Prior art: this issue looks like it could be related to https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-127?filter=allissues&orderby=priority%20DESC&keyword=transitive but that seems to have been fixed. #2020-03-26 10:45:55 dominicm I don't think repos are conveyed across deps.edn projects. They're considered local like :aliases #2020-03-26 10:47:07 daveliepmann Right, but the consumer lib doesn't know, or in my mind shouldn't care, that the lib it's requiring uses a repo that's not Maven or Clojars. #2020-03-26 10:48:37 daveliepmann Yup that looks like it, thanks. I missed it in my Jira search. #2020-03-26 10:48:54 dominicm I can think of a couple cases where that's not true. Banks being one, repos are not transparent there. There's also the risk of someone adding a repo that them causes shadowing. I'm not sure how maven works for this. #2020-03-26 11:45:42 rickmoynihan I might be misunderstanding the problem here - but isn’t it a security risk for a transitive dep to specify a new :mvn/repo; if we’re not careful might it not cause tools deps to fetch other deps from there too?
Granted 3rd party deps are always a security problem; but this suggested change in behaviour seems like it would obfuscate things further and potentially makes auditing harder?
I think the usecase is interesting; but would it not be safer for libs containing :mvn/repos to cause tools.deps to fail with a message saying “add X to :mvn/repos”? #2020-03-26 11:49:29 rickmoynihan Also couldn’t it open up new attack vectors? e.g. an attacker could possibly set it to a private repo you own; and cause you to inadvertantly package secret deps in published binaries etc? Kind of like a CSRF attack.
Admitedly this is a fanciful attack — and it would require you to depend on an attackers library either explicitly or transitively -- but attacks usually are elaborate. #2020-03-26 11:55:12 dominicm That's right. But maven allows this, so it might already be the expected model. #2020-03-26 12:22:26 alexmiller Yes, this should work, just not handled right now #2020-03-26 13:00:43 ghadi That attack vector sounds realistic #2020-03-26 13:09:34 alexmiller it needs some thought. imo, this is an under-considered part of the Maven model in general. repos are identified by name, and those names are just totally arbitrary and unique solely by rarity/convention. #2020-03-26 13:10:42 alexmiller but Maven does certainly let you add repos in transitive deps for this case. I'm not sure if their use is "scoped" to that dep and below though #2020-03-26 13:46:24 rickmoynihan yeah was wondering about this too… seems less risky to scope down the subtree; than apply globally. #2020-03-26 13:12:41 alexmiller they are somewhat shielded from this by using a top-down, breadth-first, first one found wins strategy as "parents" in the tree have already been resolved before the transitive repo is encountered #2020-03-26 13:16:39 alexmiller that's kind of also true with tools.deps (resolution happens top-down), but selection can be modified during the traversal and the "source" is not tracked right now in any way #2020-03-27 03:39:02 dorab Using Sean Corfield's fork of depstar, and running the command clj -A:uberjar x.jar, I get
Building uber jar: x.jar
{:warning "clashing jar item", :path "META-INF/services/com.fasterxml.jackson.core.JsonFactory", :strategy :concat-lines}
{:warning "clashing jar item", :path "META-INF/services/com.fasterxml.jackson.core.JsonFactory", :strategy :concat-lines}
I know it is just a warning, but what do I do to fix it? My deps.edn is
{:paths ["src"] ;until TDEPS-52 is fixed
:aliases
{:test
{:extra-paths ["test"]
:extra-deps {org.clojure/test.check {:mvn/version "RELEASE"}}}
:runner
{:extra-deps
{com.cognitect/test-runner
{:git/url ""
:sha "76568540e7f40268ad2b646110f237a60295fa3c"}}
:main-opts ["-m" "cognitect.test-runner" "-d" "test"]}}
:deps
{
cheshire {:mvn/version "5.10.0"}
http-kit {:mvn/version "2.3.0"}
org.clojure/clojure {:mvn/version "1.10.1"}
}
}
If this is not the appropriate channel, please redirect me. Thanks. #2020-03-27 03:49:17 seancorfield You don't do anything to fix it. It's just warning you, so you know that the same file is being copied into your JAR multiple times (and it is handling it by concatenating the files). Jackson is a horribly problematic library -- it is the cause of most of the version conflicts we see people run into these days but unfortunately it's also fairly ubiquitous so... #2020-03-27 03:50:07 seancorfield I'm a bit surprised you're seeing the message twice -- that implies three copies of the file are being found on the classpaths... #2020-03-27 03:52:53 seancorfield The 1.0.x version of depstar (currently on master) will have a -S / --suppress-clash option to hide that warning message(!) but you can currently use the -v / --verbose option to learn more about what it is bringing in @dorab #2020-03-27 04:01:23 seancorfield OK, I've repro'd based on your deps.edn and it looks like each of the three Jackson libraries being pulled in by Cheshire all contain that same file. #2020-03-27 04:08:00 seancorfield Yup, seems like each of the Jackson JAR files (core, databind-cbor, and databind-smile) all contain that file and it has a class name in it that matches one in the library it comes from. I'm fairly certain this is harmless and "expected behavior". Perhaps @ghadi can confirm when he is next around, since he's the original author of the first version of depstar...? #2020-03-27 04:09:18 seancorfield (this seems like another reason to try using clojure.data.json instead of cheshire 🙂 ) #2020-03-27 04:17:11 dorab Thanks @seancorfield. I will try to use clojure.data.json instead. #2020-03-27 04:21:08 dorab If the same file is being requested multiple times, could depstar just include it once? Sorry if I'm being naive. #2020-03-27 04:31:30 seancorfield No, the same named file is included in three different libraries. #2020-03-27 04:32:55 seancorfield As I said above, Cheshire pulls in Jackson, and that means you get its core, cbor, and smile libs (dataformat, not databind, my bad -- rest of it still applies). Each of those three libs includes the same file. depstar concatenates the lines in each file to create a single file in the resulting JAR. #2020-03-27 04:36:56 dorab Gotit. Thanks. #2020-03-27 04:37:33 seancorfield Interestingly, dataformat-smile and dataformat-cbor are both deprecated and archived. Ah, looks like dataformat-binary is the replacement for both of those now. Sheesh, that is one weird, troublesome library... #2020-03-27 15:47:04 fabrao Hello all, how to convert this lein uberjar
:main ^:skip-aot app.core
:uberjar-name "project.jar"
:profiles {:uberjar {:aot :all}} #2020-03-27 15:47:21 fabrao to deps.end alias? #2020-03-27 16:31:28 seancorfield It has an example in the readme
# generate pom.xml (or create it manually)
clojure -Spom
# build the uberjar with AOT compilation
clojure -A:depstar -m hf.depstar.uberjar MyProject.jar -C -m project.core
# Main-Class: project.core
java -jar MyProject.jar
#2020-03-27 16:31:52 seancorfield (use project.jar instead of MyProject.jar and app.core instead of project.core) #2020-03-27 16:33:23 seancorfield You'll need to edit the pom.xml generated by clojure -Spom to add more stuff. Take a look at a pom.xml file generated by clj-new (if you created your project with a recent version of clj-new and the app template, then the pom.xml file will probably be almost 100% of what you would need for depstar and deployment to Clojars anyway). #2020-03-27 17:22:34 fabrao @seancorfield I tried -A:uberjar from your dot-clojure but I didn´t know about pom.xml #2020-03-27 17:23:01 fabrao so I creat project.clj to generate jar #2020-03-28 18:21:02 rgm apologies if this is in the docs, but is there a way to give a path to a deps.edn, instead of assuming it's in the current working directory? #2020-03-28 18:22:37 rgm (I'm coexisting with a bunch of Python and I'd prefer to have one deps.edn someplace else and just use long Makefile commands to get at the clojure libs) #2020-03-28 18:25:10 alexmiller no #2020-03-28 18:34:33 rgm drat. #2020-03-28 18:34:48 rgm symlinks it is ¯\(ツ)/¯ #2020-03-28 18:36:19 rgm (or rather, I'll just cd other_place && clojure ... to keep my Python colleagues from tripping over the clojure stuff). #2020-03-28 18:40:18 seancorfield @rgm One thing you can try is specifying CLJ_CONFIG=/path/to/folder clojure ... where your deps.edn file is /path/to/folder/deps.edn -- this treats that deps.edn file as the user-level one, ignoring your ~/.clojure/deps.edn (and it would still pick up a deps.edn file in the current folder). #2020-03-28 18:40:46 seancorfield (so you'd get system deps, CLJ_CONFIG deps, current/project deps) #2020-03-28 18:41:06 rgm oh, that's a great trick ... I don't have a ton of deps #2020-03-28 18:41:17 rgm don't really even need a current folder one. #2020-03-28 18:41:46 rgm (which is what got me going down this path, to keep things tidy). #2020-03-28 18:41:47 seancorfield We use this approach in our monorepo at work, so we can have a "standard" user-level deps plus our current subproject deps (and ignore any developer-level user deps on different machines). #2020-03-28 18:46:19 rgm right, and because it's env, I can just manage it through settings in Makefile, or .env, or direnv, or ... this is great. #2020-03-28 18:54:32 rgm (Can I just say you and Alex are killing it, and it’s much appreciated?) #2020-03-28 19:13:39 borkdude @rgm fwiw deps.clj (an unofficial clj script re-implementation, or the binary version deps.exe) does have an option for this #2020-03-28 19:13:39 borkdude @rgm fwiw deps.clj (an unofficial clj script re-implementation, or the binary version deps.exe) does have an option for this #2020-03-28 19:41:54 rgm I hereby include you in the "killing it" set. #2020-03-29 19:04:08 jose is there an example about how to add a java file to my clojure project? In leiningen there is a :java-source-path , but I think there is nothing similar for tools-deps #2020-03-30 00:32:05 potetm @jlle You can also call javac from the cli. #2020-03-30 13:34:18 jose I found that badigeon / pbuilder are good for my use case #2020-03-30 14:12:27 borkdude $ clj -Sdeps '{:aliases {:foo {:main-opts ["-e" "(ns,foo,(:require,[clojure.java.shell,:refer,[sh]])),(sh,\"javac\",\"Foo.java\"),(shutdown-agents)"]}}}' -A:foo
😜 #2020-03-30 14:13:24 borkdude (it took me only 10 minutes to figure out the correct nesting and seancorfield-commas on the command line) #2020-03-31 16:00:00 jjttjj Is there a way to set the filename clash strategy with depstar? I'd like my data_readers.cljc to be merged #2020-03-31 16:48:04 seancorfield Feel free to open an issue on GitHub to add support for data_readers with additional file extensions. Seems like a good idea to add that. @U064UGEUQ #2020-03-31 17:12:53 jjttjj Will do! #2020-04-01 08:34:10 pez What is the equivalent of Leiningen's :local-repo "local-m2" in deps.edn? #2020-04-01 08:38:28 delaguardo :mvn/local-repo key #2020-04-01 08:58:15 pez It doesn’t seem to work as I expect it to. I don’t know what to put as the url. #2020-04-01 12:18:57 alexmiller It takes a string (a path) #2020-04-01 12:19:50 alexmiller Which can be either relative or absolute #2020-04-01 12:21:59 pez Thanks. I shall experiment some more, maybe I am not looking at the right places for the right result... Also, right now I'm using lein-tools-deps so maybe that complicates things. #2020-04-01 12:22:44 alexmiller I suspect that probably won’t work #2020-04-01 12:21:55 alexmiller This seems wrong as it is relative to the current directory. So if you are sharing it, you need an absolute path #2020-04-01 11:19:58 cursork I think I'm just having a stupid moment... But I'm not sure where I can use :jvm-opts in deps.edn . I'm currently using it in a dev alias and calling clj -A:dev which works fine. But the entire purpose of this code is to interoperate via JNI to an underlying dynamic lib, so ideally I'd like to have :jvm-opts ["-Djava.library.path=..."] always be set to a path I control.
Using System/setProperty doesn't seem to have an effect as I guess the initial java.library.path at startup is used.
Is the only option to have an alias for the path, and always invoke it? And then other aliases as usual? #2020-04-01 11:34:27 cursork Maybe I'm answering my own question... If there's a .dylib or .so I'm expecting to use, the location should be specified for each env it's deployed on. As it could be different between developer's machines as well. So maybe we all have a ~/.clojure/deps.edn with some profile with the correct path. A common name of :shared-libs or something might make that easy to work with. #2020-04-01 11:44:47 kszabo yeah, jvm-opts only works with aliases #2020-04-01 11:47:32 cursork Thanks for the confirmation 🙂 #2020-04-02 12:59:05 rfhayashi Hi! I'm trying to invoke tools.deps directly from clojure (more specifically clojure.tools.deps.alpha/resolve-deps) but I get a
Cannot access s3p://... using the registered transporter factories: HttpTransporterFactory, S3TransporterFactory, FileTransporterFactory
error. If I run a clj -Sdeps with the same content it works fine. Is there any library that I should include in the classpath to make s3 repos work? Or any environment variable or system property that I should set? I'm using org.clojure/tools.deps.alpha {:mvn/version "0.8.677"}. #2020-04-02 13:06:31 alexmiller @rfhayashi did you try s3:// ? that's the only documented protocol for s3 access (s3p did work in some older versions but has never been supported) #2020-04-02 13:07:02 alexmiller you should not have to do anything special to get s3 to work #2020-04-02 13:16:03 rfhayashi @alexmiller it does work. thanks! #2020-04-02 13:54:01 kszabo is there any support for modifying the bootclasspath, like:
https://github.com/sunng87/lein-bootclasspath-deps
does for leiningen? It would be nice to use some special case of :boot-deps so that I don’t have hack it together with downloading jar files locally from maven and using :jvm-opts directly #2020-04-02 13:54:46 kszabo this seems to be in the focus area of tools-deps of invoking java with the correct dependencies ready #2020-04-02 14:02:30 alexmiller you should not modify the bootclasspath in my opinion, so I'm not going to make that any easier #2020-04-02 14:14:15 alexmiller you can make an alias that produces the boot classpath you need, then use clj -Spath to generate the classpath from it, so there is no need for manual jar downloading #2020-04-02 14:14:58 alexmiller and then put that path in your :jvm-opts #2020-04-02 14:15:46 kszabo clj -Spath would produce an absolute path to my main maven repo, which is hosted in my home directory #2020-04-02 14:16:06 kszabo meaning the resulting :jvm-opts needs to be scripted on each machine #2020-04-02 14:16:51 alexmiller an alternative would be to set :mvn/local-repo to fetch it into a subdirectory of the project, then use relative paths in the jvm-opts #2020-04-02 14:17:19 kszabo wouldn’t that apply to all maven deps? I haven’t used that one yet #2020-04-02 14:17:42 kszabo I don’t want to download maven again 😂 #2020-04-02 14:18:11 alexmiller you would just use this for what you need in the bootclasspath #2020-04-02 14:19:16 kszabo I see #2020-04-02 14:19:20 kszabo thanks for the help #2020-04-02 21:29:41 Jaime Sangcap I'm using clojure cli / deps.edn. Is there a way to define env vars following the 12 factor app?
Looking into https://github.com/weavejester/environ but it seems like it does not support deps.edn (for instance, if your using lein. you can have .lein-env) #2020-04-02 21:35:53 seancorfield @jaime.sangcap (System/getenv "varname") is all you need to get an environment variable's value so I'm not sure what you're asking in the context of CLI/`deps.edn`? #2020-04-02 21:36:38 seancorfield Perhaps "Can you have environment variables in your deps.edn?" No. It's a pure EDN file, not executable code, and has no substitutions made. #2020-04-02 21:43:01 Jaime Sangcap I'm wondering if there is library that can load env var from file. And you can configure where the file is in the deps.edn.
Right now I'm only using intellij to set env vars.
What I'm thinking is there should be a example file (checkin to version control) that has all the required env var
Then locally, the library will check the example file against your env file for missing env var #2020-04-03 16:21:56 Jaime Sangcap cool! Yes I think can use this. Thanks 🙂 #2020-04-02 21:44:01 Jaime Sangcap Before loading them and throw error is you missed to set something #2020-04-02 21:44:14 seancorfield Sorry, I have no idea what you mean by any of that. #2020-04-02 21:45:33 Jaime Sangcap Thanks! I think I'm mixing different things #2020-04-02 21:46:44 seancorfield Environment variables are set at the O/S level before a process is started -- I would imagine IntelliJ is setting the environment variables before starting the REPL or run process. #2020-04-02 21:47:44 seancorfield You can't set environment variables from inside a process. You can set JVM properties tho'... #2020-04-02 21:49:01 seancorfield BTW environ is just a library. It can be used from any process run by CLI/`deps.edn`. #2020-04-02 21:49:21 seancorfield It just happens to have existing plugins for lein and boot that call it in particular ways. #2020-04-02 21:54:08 seancorfield (but I would caution you against storing anything in deps.edn that isn't part of the official syntax -- deps.edn is not intended to be a "general configuration file -- which project.clj and build.boot tend to be) #2020-04-03 07:38:24 practicalli @jaime.sangcap Storing environment variables in a file that you then version control is the best way to add a huge security hole in your application and isn't following 12 factor app approach.
If you are using a service like Heroku, then that service manages the Environment variables securely for each application and you can the just read them in from Clojure as Sean mentioned or using environ. #2020-04-03 08:56:11 Jaime Sangcap I agree. Thank you for pointing it.
However, what I meant was commit in version control an example file. Let's say .env.example , this will contain all the env vars that are needed by your application.
APP_PORT=8000
DATABASE_URL=
AWS_ACCESS_KEY_ID
SOME_PATH=/temp
Notice that env vars don't have values, but are there so that when other devs can copy the .env.example and create .env and provide the values.
I think an advantage of this one is you can check the .env file if there are missing keys from .env.example and throw an error before the app starts to prevent it from running in unpredictable state. #2020-04-03 07:41:58 teodorlu Option: store config defaults in an .edn file, and create a custom read-config function that reads an env var if it is present, otherwise defaults to the file.
Then you can set defaults, and override them with environment variables.
Note: env vars are named by strings. We often like to use keywords as identifiers in Clojure. You might want to set a naming scheme that lets you query with keywords rather than strings. #2020-04-03 08:28:17 dominicm Sounds like aero :) #2020-04-03 09:37:27 teodorlu Aero seems to make all of this possible to do from within the config file. README[1] excerpt:
{:port #or [#env PORT 8080]}
[1]: https://github.com/juxt/aero #2020-04-03 09:52:05 Jaime Sangcap @seancorfield Yes I think I got that one. Given I have this deps.edn
{:deps
{org.clojure/clojure {:mvn/version "1.9.0"}}
:paths ["src"]
:aliases {:dev {:extra-paths ["dev"]
:extra-deps {nrepl/nrepl {:mvn/version "0.6.0"}}
:main-opts ["-m" "nrepl.cmdline"]}}}
I could imagine intellij when you add env var DATABASE_URL to it, it will just run using DATABASE_URL=jdbc-url clj -A:dev
Looking at environ code, all functions are private 🙂
Maybe I could just do this. (copied from this convo https://github.com/weavejester/environ/issues/14)
(require '[environ.core :as environ])
(defonce env (merge (read-my-custom-config-file) environ/env)
Where config file is in .gitignore and will be used in local dev.
Thanks to @teodorlu for the idea #2020-04-03 12:35:21 alexmiller I’ve done something very similar to that in a real app #2020-04-03 13:28:25 jjttjj Is there a "recommended" way to have a repl start in a particular namespace and load that file? #2020-04-03 13:29:05 jjttjj I've been trying various arrangments of this but none seem to really be what I'm going for
:main-opts ["-i" "src/jt/jt15.clj"
"-e" "(in-ns,'jt.jt15)"
"-r"] #2020-04-03 13:31:11 jjttjj nevermind, had a typo in the namespace, this is working #2020-04-04 00:14:53 cfleming How does deps handle snapshot versions? There doesn’t seem to be any setting for how often repos should be checked for new snapshots. Is there even any concept of a snapshot version in deps? #2020-04-04 00:44:11 seancorfield I thought the underlying Maven library checked snapshots once a day? #2020-04-04 00:45:42 cfleming Deps doesn’t seem to have a knob for that and I’m not sure what the default is. #2020-04-04 01:09:36 hiredman I think snapshot builds may be fundamentally incompatible with how tools deps caches the built classpath #2020-04-04 01:11:12 hiredman Or, I guess what is more likely is tools deps checks every time it needs to rebuild the cache, but if the classpath is cached and doesnt need rebuilding you'll never get a new snapshot #2020-04-04 14:20:29 alexmiller This #2020-04-04 01:21:19 seancorfield Hmm, yeah, that makes sense. I guess the "knob" there is that you can -Sforce a rebuild of the classpath. I know Alex has said in the past that "RELEASE" and "LATEST" fly in the face of the stability that deps.edn is aiming for... #2020-04-04 05:20:53 cfleming Yeah, I did wonder if deps might be philosophically opposed to them. If hiredman is correct that explains what some people are seeing in Cursive (very slow launches of REPLs) since Cursive doesn’t use the classpath caching. #2020-04-04 13:46:48 ghadi RELEASE & LATEST and origin/master are mutable references
SHAs are values #2020-04-04 14:16:05 alexmiller Snapshots are checked daily by default and there is no knob to change that atm #2020-04-04 14:17:12 alexmiller -Sforce will NOT force a recheck (although I’ve thought about making that flag also trigger update always so it would) #2020-04-04 14:17:55 alexmiller The only real way to force a snapshot to update right now is to rm from your m2 repository AND -Sforce #2020-04-04 14:19:52 alexmiller I don’t see how snapshots would be any more likely to cause slowness than any other version #2020-04-04 20:13:09 cfleming This can happen with lein if it’s configured to check for new versions always. That means every project resolve will hit the remote repo for every snapshot dep. #2020-04-04 23:46:30 alexmiller Ah, yeah. Should just be daily w deps #2020-04-04 19:39:00 athomasoriginal Compiling a CLJS program and running a CLJS repl is straightforward enough with
clj --main cljs.main --compile ns.name.here --repl
However, if I want to run a cljs socket repl so I can connect my editor to it (Atom) I cam up with these two options
Option 1
; compile cljs
clj --main cljs.main --watch "src" --compile ns.name.here
; run a socket repl
clj -J-Dclojure.server.browser="{:port 4444 :accept cljs.server.browser/repl}"
Option 2
; do both in one command
clj -J-Dclojure.server.browser="{:port 4444 :accept cljs.server.browser/repl}" -- --main cljs.main --watch "src" --compile ns.name.here
I’m now wondering if there is a better option which I missed? 🙈 #2020-04-05 07:55:39 didibus One thing I once in a while wish for is a way to have tools.deps list out all of the available aliases. #2020-04-05 07:57:12 didibus Like it be nice if clj -A would list out the available alises instead of throwing an error. And same thing could happen for all other alias type, clj -R to list available resolve-deps alias, etc. #2020-04-05 10:38:45 borkdude @didibus since it's EDN it's very easy to do this yourself? #2020-04-05 13:52:48 alexmiller I like that idea #2020-04-05 13:53:59 alexmiller Let me think about the impl a little #2020-04-05 17:11:04 vlaaad I'm pretty sure Cursive does that #2020-04-05 22:05:38 cfleming Yeah, Cursive shows all aliases and also whether they’re system or project ones. It doesn’t distinguish between system and user at the moment since the only way to determine that currently with deps is counting the config files, which is a little gross. #2020-04-06 11:21:21 pez I am trying to move a project over from Leiningen to deps. In the project.clj there is a top level :exclusionsvector. Anyone familiar enough with both lein and deps to hint me where I should look up how to translate it to deps? #2020-04-06 12:27:10 alexmiller There is no support for global exclusions right now #2020-04-06 12:27:51 alexmiller But you can repeat it at each top level dep if needed #2020-04-06 12:32:03 pez Thanks! We'll see if needed. 😃 #2020-04-06 11:26:16 vlaaad hmm, I think I saw something about using :override-deps with nil values… #2020-04-06 11:30:37 mgrbyte I am trying to figure out how to create an uberjar with exlusions (This achieved with the :provided profile in lein). I'm using depstar atm. Has anyone found a way to do this? #2020-04-06 11:45:42 rickmoynihan @pez: it’s translates directly:
some/dep {:mvn/version "3.0.0"
:exclusions [ch.qos.logback/logback-classic]} #2020-04-06 12:22:40 teodorlu Note: deps uses different dependency resolution from Lein (which relies on Maven). It might make sense to check if you need the exclusions after moving to deps.
The ClojureScript podcast S3E4[1] with @alexmiller provides some discussion on this.
[1]: https://clojurescriptpodcast.com/ #2020-04-06 12:27:22 imre Would it go against deps philosophy to add support for something like provided? To me that (although it could perhaps be better named) communicates for (some of) its functionality, this project needs xx/yy, however it is left to the consumer to provide those. Or is a dev alias with those dependencies the preferred way? #2020-04-06 12:28:27 alexmiller The latter #2020-04-06 12:30:32 imre Understood, thanks #2020-04-06 12:30:38 pez @rickmoynihan yes, those exclusions work. This is a top-level key that Leiningen supports. I am not even sure what it does, only that I fail to get my project running and thinking it might be this. But after trying with @vlaaad’s suggestion, indicates I also have some other error in my config. #2020-04-06 16:52:30 seancorfield @pez FWIW, we used exclusions heavily with both lein and boot but once we switched to deps.edn we were able to get rid of nearly all our exclusions (I think we still have a couple of exclusions of ClojureScript in our backend deps because we just don't need/want that brought in). #2020-04-06 17:10:40 pez @seancorfield, Thanks! I'll listen to that podcast that @teodorlu mentioned and see if I can figure out why. 😃 #2020-04-06 21:24:48 seancorfield @mikerod Recent commits have made add-lib depend on Java code in that repo so you can no longer depend on it via git. I built a local version to use, so I could work with the most recent features & fixes. Alex has talked about some form of add-lib ending up in core or part of the REPL or something... all TBD at this point. #2020-04-06 21:25:47 mikerod Ah ok. Thanks for the details. #2020-04-06 21:25:54 seancorfield My "dot-clojure" file (`deps.edn`) has a :deps alias that depends on a git SHA prior to the Java code dependency. #2020-04-06 21:26:03 mikerod I did see that one. #2020-04-06 21:26:15 mikerod But guess it requires some more. #2020-04-06 21:27:13 seancorfield Well, it can be used as-is -- and it generally works just fine. I used it quite a bit in that form for loading new libraries into my running REPL (since I tend to start a REPL and run it for days). #2020-04-07 14:49:15 mikerod I believe I read there were issues in newer JVMs around trying to add to URL classloaders or something of that sort. This has been worked around in this branch? I’m going to look a bit at the impl because I’m curious what is being done. #2020-04-07 14:50:24 mikerod (which is used by pomegranate) #2020-04-07 15:58:52 seancorfield @mikerod If you create a dynamic class loader, you can load new libraries/classes into it: that's how we embed Clojure at work into a legacy app and how add-lib works (you can only use it in a context that has a DCL -- for example, Cognitect's REBL does not, so at work we have a small Clojure script to start REBL's UI after setting up a DCL). #2020-04-07 16:04:53 mikerod ah ok. that makes sense - thanks for the extra details #2020-04-06 21:28:26 seancorfield (I'm using a variant of 0.8.667 built locally now) #2020-04-06 21:36:14 alexmiller I'm actually working on it RIGHT NOW :) #2020-04-06 21:36:39 alexmiller Rich and I spent an hour this morning talking about it #2020-04-06 21:38:55 andy.fingerhut Quick. Everybody don't bother Alex! 🙂 #2020-04-06 21:44:19 seancorfield I deliberately didn't @ you so I wouldn't disturb you 🙂 #2020-04-06 22:02:56 mikerod Oh nice! #2020-04-06 22:03:06 mikerod Ok. I appreciate all the input. #2020-04-07 08:33:21 penryu Is there anything in tools.deps that approximates lein's :java-source-paths? #2020-04-07 08:36:47 vlaaad @penryu no, tools-deps does not compile java, it only builds class paths and runs java programs #2020-04-07 08:37:29 vlaaad you can compile java separately, release it to clojars and use as a maven dependency 🙂 #2020-04-07 08:39:39 penryu Thanks, @vlaaad. In this case, I'm going to implement some of the core functions in java, and provide a clojure wrapper. #2020-04-07 08:39:50 penryu Ah, I'll take a look at badigeon #2020-04-07 19:35:14 teodorlu Is it possible to override :main-opts from the command line? #2020-04-07 19:46:34 teodorlu Context: standard :test:runner from Sean's clj -A:new setup. Trying to make Docker download test dependencies into the image without running stuff early on to improve caching. #2020-04-07 20:01:49 alexmiller main-opts concat rather than override #2020-04-07 20:01:58 alexmiller so no if I understand your question #2020-04-07 20:02:11 alexmiller but you can always make a new alias that does what you want #2020-04-07 20:08:05 teodorlu That works. Thanks! #2020-04-10 21:47:36 seancorfield seancorfield/depstar {:mvn/version "1.0.94"} -- now merges data_readers for cljs and cljc extensions; adds -S/`--suppress-clash` option to suppress file clash warnings (some people find these information messages too annoying 🙂 ) https://github.com/seancorfield/depstar #2020-04-10 21:48:28 seancorfield seancorfield/clj-new {:mvn/version "1.0.199"} -- mostly documentation updates; generated projects now use latest dependencies. https://github.com/seancorfield/clj-new #2020-04-12 11:50:53 afhammad Hi, having issues importing java classes (generated by protobuf) from another folder in the monorepo using deps.edn . I've tried both adding the paths to :paths of my project and creating a deps.edn in the protos folder and adding that as a local dep. In both cases the classpath shows up when doing clj -Spath but starting a repl fails with ClassNotFoundException #2020-04-12 13:51:43 alexmiller If you have Java files, you need to javac compile them into classes #2020-04-12 13:52:50 alexmiller If you have classes, what you say above should work, but make sure the classes are in a directory structure that matches their package name (just like Clojure source) #2020-04-12 19:11:54 afhammad Thanks @alexmiller. Precompiling worked. Is there a nice way of doing it as part of the deps.edn setup? One reason is it requires passing in a .jar dependency in order to compile which is already a dep of the project, but having the javac compilation as a separate step means I have to worry about fetching that dependency myself. #2020-04-12 19:26:57 alexmiller it will always be an extra step, but there are some helper dep tools out there to do that #2020-04-12 19:27:29 alexmiller one option is to portion the java stuff into its own project that you can build (with mvn or something else), deploy it, and depend on it #2020-04-12 19:29:30 afhammad yeh, thats one way to go, thanks #2020-04-16 07:05:40 flipmokid Hi. What's the best way of adding java source to a deps.edn project? I have one java file that will be used as the entrypoint to the code that will call into the clojure code. It's there because Azure Function lib uses reflection on the method's arguements to determine how to convert data before passing it in and looks at the parameterised types used. #2020-04-16 07:09:49 andy.fingerhut I do not know the best way, as I haven't personally used this in a project before. I can tell you that this project claims to support the compilation of Java source files in a project that uses deps.edn. Exactly how, and whether it suits your purposes, is best determined by you: https://github.com/EwenG/badigeon #2020-04-16 09:42:01 flipmokid Thanks @U0CMVHBL2 I'll give it a try. #2020-04-16 16:38:17 flipmokid Thanks @U097654L8, that looks really good. #2020-04-18 10:39:12 Eamonn Sullivan Hi all, I have a small experimental Clojure library (https://github.com/eamonnsullivan/github-search) that I'd like to turn into a mono-repo that produces both a jar and an npm package (shadow-cljs), with some common code (.cljc). Is this possible in tools-deps? Has anyone tried this and can point me to an example I can study? #2020-04-21 07:08:28 rberger What would the equivaliant to the function leiningen.core.main/abort be in a program started by clj tool? My googling for a solution has been unsuccessful. #2020-04-21 07:17:54 dominicm I don't know the function, but I'd guess System/exit or throw based on the name. #2020-04-21 07:18:01 dominicm What behavior do you want? #2020-04-21 07:22:29 rberger I guess Print msg to standard err and exit with a value of 1. And propler terminate anything and supply proper messages... I did use System/exit for now. I’ve been translating software that use to use lein to use tools.deps toolchain and looking for things like this. #2020-04-21 11:15:15 practicalli I have shared my user level ~/.clojure/deps.edn file, which includes aliases for all the common tools I use with Clojure projects. Instructions for using the aliases are in the README and in the relevant :alias sections. For some aliases that have optional arguments I have also created additional alias names to capture the most common options, to save the amount of details that need to be remembered. Any other useful aliases are most welcome and thanks to Sean Corfield for his collection of aliases.
https://github.com/practicalli/clojure-deps-edn
Thank you. #2020-04-21 16:00:45 seancorfield @jr0cket Is yours a superset of mine at this point, or is a selection of mine (plus your own additions)? Maybe I'll mine yours for extra stuff to add to mine 🙂 #2020-04-21 17:26:31 practicalli There is a substantial cross-over of libraries includes in the aliases. You have some I have not tested yet. I have some not included.
There is some divergence in naming of aliases.
Happy to work on convergence if there is interest / value. #2020-04-21 19:35:47 jrwdunham Hi all. Looking for some help here. I'm using tools-deps and my deps.edn file has a :repl alias with an :override-deps that we can summarize as {tegere {:local/root "/path/to/tegere" :deps/manifest :deps}}. The problem seems to be that the transitive dependencies of my local deps are not present in my classpath. I can see by running clj -R:repl -Stree that I have a dependency path like tegere/tegere > com.cemerick/pomegranate 1.1.0 > org.apache.maven/maven-resolver-provider 3.5.3 > javax.inject/javax.inject 1. However, if I ls -l /root/.m2/repository/ I can see that my top-level :deps dependencies are present but the dependencies of my local [:repl :override-deps] (e.g., javax.inject, pomegranate) are not. Any advice? Thanks in advance. #2020-04-21 19:47:05 alexmiller can you provide your clj version (`clj -Sdescribe`), a deps.edn to repro, and clj -Stree output? #2020-04-21 19:57:15 jrwdunham bash-5.0# clj -Sdescribe
{:version "1.10.1.502"
:config-files ["/usr/local/lib/clojure/deps.edn" "/root/.clojure/deps.edn" "deps.edn" ]
:config-user "/root/.clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/usr/local/lib/clojure"
:config-dir "/root/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""}
bash-5.0# cat deps.edn
{:deps
{cheshire {:mvn/version "5.8.1"}
clojure.java-time {:mvn/version "0.3.2"}
clj-http {:mvn/version "3.10.0"}
com.pubnub/pubnub-gson {:mvn/version "4.25.0"}
com.taoensso/timbre {:mvn/version "4.10.0"}
nrepl/nrepl {:mvn/version "0.6.0"}
cider/cider-nrepl {:mvn/version "0.22.0"}
hickory {:mvn/version "0.7.1"}
org.clojure/tools.namespace {:mvn/version "1.0.0"}
vbclient
{:git/url " #2020-04-21 19:57:15 jrwdunham #2020-04-21 20:03:52 alexmiller and clj -Spath doesn't include pomegranate and javax.inject? #2020-04-21 20:06:08 alexmiller oh, did you try -Sforce to force re-create your classpath? #2020-04-21 20:06:43 alexmiller changes in local deps' deps.edn files do not trigger a classpath cache refresh so you might be seeing something old there #2020-04-21 20:10:51 jrwdunham Those deps are present when I run clj -Spath #2020-04-21 20:10:55 jrwdunham $ clj -R:repl -Spath
src:/root/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:...
#2020-04-21 20:11:26 jrwdunham but there is nothing at those locations on disk #2020-04-21 20:11:49 jrwdunham bash-5.0# ls /root/.m2/repository/javax
ls: /root/.m2/repository/javax: No such file or directory
#2020-04-21 20:19:00 jrwdunham Odd, if I run clj -R:repl -Sforce -Stree then it seems to work... #2020-04-21 20:19:55 jrwdunham bash-5.0# ls /root/.m2/repository/javax
inject
#2020-04-21 20:20:42 jrwdunham thing is, I'm trying to build a Docker image, and having the following RUN statements in my Dockerfile is not having the same effect:
RUN clj -R:repl -Sforce -Stree
RUN clj -R:repl -Sforce -Stree
#2020-04-21 21:23:57 jrwdunham Well, after removing the .cpcache/ dirs on my host machine and ensuring I have a Docker COPY that ensures my local :override-deps paths are available at build time (d'oh!), and running a Docker build with --no-cache, all seems to be working as expected. #2020-04-21 21:24:58 jrwdunham Thanks for your help @U064X3EF3 (even if the issue seems ultimately to have been my Docker misconfiguration)—I at least learned more about how to use tools-deps, and maybe this will help somebody else. #2020-04-21 21:26:09 alexmiller if you have a cached classpath, it does not currently verify that the dirs/files on the classpath actually exist, so manual mucking with ~/.m2 (either by the filesystem or your container config) can lead to some confusing results. -Sforce will usually help you there #2020-04-21 21:26:44 jrwdunham cool. gtk #2020-04-21 21:27:00 alexmiller it's a little tricky to implement this check as it would need to be in the bash script #2020-04-21 21:27:20 alexmiller and it requires some interpretation b/c missing path dirs is fine and pretty common #2020-04-22 16:32:49 unbalanced Okay finally have my private maven all set up on artifactory, verified that the code is deployed there. Choking on how to include that in the deps.edn for the project, though.
:deps
{com.business.projname {:mvn/version "0.1.5"
:mvn/repos {"central" {:url ""}}}
I assume that is not correct #2020-04-22 16:33:47 unbalanced on
$ clj -m figwheel.main -bo dev
Error building classpath. Could not find artifact com.business.projname:com.business.projname:jar:0.1.5 in central ()
#2020-04-22 16:36:31 unbalanced ah, think I got it. well at least it's complaining it can't file the file at the right location #2020-04-22 16:36:37 unbalanced new error message, I'll call that a success #2020-04-22 16:37:14 imre Isn't :mvn/repos supposed to be a top-level key? #2020-04-22 16:38:28 unbalanced yep. and I needed so use deps correctly #2020-04-22 16:39:15 unbalanced :mvn/repos {"business" {:url ""}}
:deps {com.business/projname {:mvn/version "0.1.5"}}
#2020-04-22 16:39:21 unbalanced now at least I get a new error with that #2020-04-22 16:39:22 unbalanced 😄 #2020-04-22 16:41:43 unbalanced ah okay, no error message now, works exactly as desired #2020-04-22 16:41:44 unbalanced yay #2020-04-22 16:41:58 unbalanced suspiciously painless #2020-04-23 06:11:12 ikitommi when adding dependency using :git/url, using clojure.tools.namespace.repl/refresh seems to refresh also namespaces from those dependencies. Is there a way not to do that? #2020-04-23 06:12:55 ikitommi added dependency to malli and it pulled sci, refreshing a lot of non-project ns’es:
:reloading (sci.impl.namespaces sci.impl.utils sci.impl.destructure sci.impl.fns sci.impl.interop sci.impl.doseq-macro sci.impl.exceptions sci.impl.max-or-throw sci.impl.for-macro sci.impl.analyzer sci.impl.readers sci.impl.parser sci.impl.interpreter sci.core malli.core malli.provider malli.json-schema malli.transform malli.edn malli.error malli.mermaid malli.swagger user sci.impl.main malli.util malli.generator)
#2020-04-23 06:25:27 alexmiller I don't know enough about refresh to say #2020-04-23 06:26:58 alexmiller I think refresh looks at directories on the Java classpath, and git deps will cause dep paths to be there #2020-04-23 06:27:09 alexmiller refresh doc string says "The directories to be scanned are controlled by 'set-refresh-dirs'; defaults to all directories on the Java classpath." #2020-04-23 06:27:24 alexmiller so you could use that to narrow the scope #2020-04-23 06:28:11 alexmiller (clojure.tools.namespace.repl/set-refresh-dirs "src")
#2020-04-23 06:28:14 alexmiller might be sufficient #2020-04-23 06:28:42 ikitommi when using :mvn dependency, the code is in jars and because of that, not in scope of refreshing? #2020-04-23 06:29:14 alexmiller right #2020-04-23 06:30:14 ikitommi set-refresh-dirs did the trick, thanks! #2020-04-23 06:30:32 alexmiller when in doubt, read the docs :) #2020-04-23 06:50:46 ikitommi would be great if this worked out-of-the-box, e.g. convention from DEPS to which folder the pulled deps are in, that is automatically excluded in TNS. #2020-04-23 07:06:36 alexmiller by "box" I assume you mean tools.namespace? #2020-04-23 07:07:25 alexmiller I think I would invert it - only include src dirs that are relative to the current directory and exclude others #2020-04-23 07:07:41 alexmiller tns doesn't know anything about deps or git deps or whatever #2020-04-23 08:29:01 holyjak Hi! Is there support for adding an optional dependency, without specifying its version? clj-http has an optional dependency on cheshire and I would like to include it, using whatever version this version of clj-http wants. So instead of cheshire/cheshire {:mvn/version "5.8.1"} I want just cheshire/cheshire . Is this possible? Thank you!!! #2020-04-23 11:27:20 borkdude I don't think it's possible since clj-http doesn't prescribe the cheshire version as a dependency, it's only available at development time for clj-http I think? #2020-04-23 12:28:37 alexmiller No, this is not possible, you’ll need to pick a version #2020-04-23 14:55:06 Laverne Schrock I feel like I've heard mention of others running into this:
Cloning: <actual repo name snipped>
Error building classpath. Destination path "ops-cli" already exists and is not an empty directory
org.eclipse.jgit.api.errors.JGitInternalException: Destination path "ops-cli" already exists and is not an empty directory
at org.eclipse.jgit.api.CloneCommand.verifyDirectories(CloneCommand.java:257)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:189)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:49)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
at clojure.tools.gitlibs.impl$ensure_git_dir$fn__955.invoke(impl.clj:109)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:105)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
at clojure.tools.deps.alpha.extensions.git$eval999$fn__1001.invoke(git.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:169)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:237)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:217)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:55)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:47)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:81)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:72)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:101)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:95)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:146)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:118)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
Unfortunately, when I was trying to figure out what the reproduction steps were, the problem suddenly went away. I can however say, that before it started working I was able to confirm that ~/.gitlibs/libs/.../ops-cli existed with a few SHA directories, but not the one corresponding to the one I was using in my deps.edn. This was running version 1.10.1.492 of clj/clojure on Linux.
Would it be possible to add some logging here: https://github.com/clojure/tools.gitlibs/blob/master/src/main/clojure/clojure/tools/gitlibs/impl.clj#L109 so that in the future we can see what the original Throwable was? #2020-04-23 15:01:52 alexmiller I have some good guesses at this point what the problem is #2020-04-23 15:02:39 alexmiller I think it's related to having two threads/processes cloning the same lib in parallel #2020-04-23 15:03:03 alexmiller because deps expansion is now parallel this is much easier to encounter now #2020-04-23 15:03:36 alexmiller and/or it could even just be bugs in the much older version of jgit we're using #2020-04-23 15:04:28 alexmiller I have a long paused set of work in a branch to update to latest jgit etc and I wanted to do that first #2020-04-23 15:05:32 alexmiller if it is the parallel stuff, you can set -Sthreads 1 to turn it down #2020-04-23 15:53:46 Laverne Schrock I'll give that a try if I run into it again #2020-04-23 15:06:36 alexmiller probably longer term to actually "fix" this, needs more thought #2020-04-24 05:44:02 lispyclouds Hello, what is a recommended/nice way of organizing a mono-repo like project of multiple tools.deps backed projects in terms of being able to run all tests and produce JARs/docker images/run tests for each from the root of the project? I have something like this, using Makefiles:
.
├── Makefile
├── docker-compose.yaml
├── consumer
│ ├── Dockerfile
│ ├── Makefile
│ ├── deps.edn
│ ├── src
│ │ └── queue
│ │ └── consumer.clj
│ └── test
│ └── queue
│ └── consumer_test.clj
└── producer
├── Dockerfile
├── Makefile
├── deps.edn
├── src
│ └── queue
│ └── producer.clj
└── test
└── queue
└── producer_test.clj #2020-04-24 06:33:56 dominicm @rahul080327 I'd recommend looking at juxt edge which is my implementation of a monorepo. If you are looking to share makefiles, you probably want the include instruction in make (not shown in edge) #2020-04-24 06:53:20 lispyclouds This looks really neat @U09LZR36F!
I was considering going the shell approach but was thinking the need for manual orchestration and error handling might be complex hence opted for Make. But will try out your organized approach! Thanks! 😄 #2020-04-24 06:34:48 dominicm In edge I solve the reuse problem with bin scripts at ../bin/eftest, etc #2020-04-24 13:58:41 zane Is it possible to use clj and friends private repositories via a password-protected SSH key? #2020-04-24 14:00:04 ghadi there's no interactive ssh with the tools @zane #2020-04-24 14:00:43 ghadi ssh-add the key to your running ssh-agent #2020-04-24 14:00:50 ghadi then you can run clj #2020-04-24 14:02:30 zane @ghadi Yes, I did that but I’m getting an error (`USERAUTH fail`) when I try it with a password-protected SSH key. #2020-04-24 14:03:20 zane Works when I use a SSH key that does not have a password. #2020-04-24 14:03:38 ghadi if you run ssh-add -l you see the key you intended to add? #2020-04-24 14:04:24 zane I believe I added it, yes. Let me run through it one more time to be sure. #2020-04-24 14:04:47 zane But it sounds like what you’re telling me is that this ought to work. 🙂 #2020-04-24 14:04:59 alexmiller I don't think it should work #2020-04-24 14:05:23 alexmiller I don't think password protected ssh keys are supported in the set of stuff we're running #2020-04-24 14:05:23 zane Ah, okay. #2020-04-24 14:05:26 ghadi totally should work #2020-04-24 14:05:43 ghadi it prompts for pw when adding to the ssh-agent #2020-04-24 14:05:50 ghadi it's non-interactive after that #2020-04-24 14:06:18 ghadi if you see the key in your agent, make sure it doesn't appear in ~/.ssh/config #2020-04-24 14:06:58 ghadi JSch, the underlying ssh library, is stupid w.r.t. understanding .ssh/config when there is also an ssh-agent present #2020-04-24 14:07:23 ghadi (It prefers to read the key directly from the file, if it knows about it) #2020-04-24 14:07:31 ghadi (Rather than consulting the agent) #2020-04-24 14:08:12 ghadi So, what's in your ssh agent (`ssh-add -l`) and what's in your .ssh/config? #2020-04-24 14:09:14 zane % ssh-add -l
4096 SHA256:… /Users/zane/.ssh/id_rsa (RSA)
% cat ~/.ssh/config
Host *
AddKeysToAgent yes
UseKeychain yes
IdentityFile ~/.ssh/id_rsa
#2020-04-24 14:09:29 ghadi ok comment out that block in your .ssh/config #2020-04-24 14:09:39 ghadi temporarily #2020-04-24 14:09:56 ghadi then run clj again #2020-04-24 14:10:23 zane clj succeeds, but I think it’s pulling from a cache. #2020-04-24 14:10:48 zane i.e. It’s using the copy of the repository I downloaded when I was using a passwordless SSH key. #2020-04-24 14:11:08 ghadi remove that from ~/.gitlibs to verify #2020-04-24 14:11:10 zane Trying to remember how to clear that cache. #2020-04-24 14:11:15 zane Rad. Thanks. #2020-04-24 14:11:23 ghadi should be in two places in there #2020-04-24 14:11:42 ghadi ~/.gitlibs/{_repos and libs} #2020-04-24 14:11:54 zane Any harm in just blowing away ~/.gitlibs entirely? #2020-04-24 14:12:16 ghadi dunno how much you have in there #2020-04-24 14:12:27 ghadi but anything will be redownloaded if you have creds #2020-04-24 14:12:42 ghadi no worse than blowing away ~/.m2/repository #2020-04-24 14:13:02 zane Awesome. #2020-04-24 14:13:11 ghadi so did it work? #2020-04-24 14:13:17 zane Trying. #2020-04-24 14:13:44 zane Started up instantly again. #2020-04-24 14:13:49 zane That’s odd. #2020-04-24 14:14:00 zane i.e. I didn’t see it download the private repository. #2020-04-24 14:14:02 ghadi clj -Sforce #2020-04-24 14:15:07 zane Worked. #2020-04-24 14:15:28 ghadi Nice. #2020-04-24 14:15:31 ghadi @alexmiller #2020-04-24 14:15:58 zane Thanks! #2020-04-24 14:16:18 zane So, what should I do with ~/.ssh/config now? #2020-04-24 14:16:23 zane Leave that stuff commented? #2020-04-24 14:16:44 zane I confess I don’t understand SSH well enough fully understand the implications of that. #2020-04-24 14:17:12 ghadi I don't think in your case there is much harm in removing just the IdentityFile portion of that block #2020-04-24 14:17:29 zane Got it. I had hoped that was the case. #2020-04-24 14:17:30 ghadi because ~/.ssh/id_rsa is one of the default searched keys #2020-04-24 14:18:03 zane But for people with multiple SSH keys that might be an issue, sounds like? #2020-04-24 14:18:20 zane (I’m writing up documentation on how to do this for the rest of my team.) #2020-04-24 14:20:59 ghadi it depends #2020-04-24 14:21:19 ghadi do you have 1 SSH key <> per repo? #2020-04-24 14:21:38 ghadi or 1 SSH key that grants you access to a bunch of repos, but you also have unrelated SSH keys present too? #2020-04-24 14:21:54 ghadi The former is not supported, the latter is, AFAICT #2020-04-24 14:22:11 zane I personally only have this one key, but I imagine others on the team might have more. #2020-04-24 14:22:14 zane I see. #2020-04-24 14:23:31 ghadi My personal recommendation is to not use files at all, only put your ssh keys into the agent #2020-04-24 14:23:57 zane I see. I’ll read up on how to do that. #2020-04-24 14:24:19 ghadi clj can use fancy newer keys like ed25519 keys as long as JSch is asking the agent to deal with them #2020-04-24 14:24:35 ghadi If JSch is trying to read them from the key files directly -> no bueno #2020-04-24 14:24:40 zane I see! #2020-04-24 14:25:07 ghadi all of this is a pain IMHO #2020-04-24 14:25:39 ghadi github style https auth tokens are a simpler model, and offer finer grained authority #2020-04-24 14:25:54 zane But that’s not supported at the moment? #2020-04-24 14:25:55 ghadi easier rotation/revocation #2020-04-24 14:26:04 ghadi right, not supported #2020-04-24 14:26:13 zane Got it. #2020-04-24 14:26:27 zane This is extremely helpful, @ghadi. Thanks again for your time. #2020-04-24 14:26:46 zane I may try to write this up so others can benefit. #2020-04-24 14:26:55 ghadi a buddy at Github told me <35% of the auth they see is ssh based #2020-04-24 14:27:31 zane Right. I certainly wasn’t using SSH with GitHub until I had to get clj to work with a private repository. #2020-04-24 14:38:39 dominicm Weird, I always considered https the less-secure approach. #2020-04-24 14:41:24 ghadi tokens can be ephemeral, ssh identities are long-lived #2020-04-24 14:45:14 ghadi good pros/cons ^ #2020-04-26 10:29:56 thheller clj -A:dev
Program 'java.exe' failed to run: The filename or extension is too longAt
C:\Users\thheller\Documents\WindowsPowerShell\Modules\ClojureTools\ClojureTools.psm1:353 char:5
+ & $JavaCmd @JvmCacheOpts @JvmOpts "-Dclojure.libfile=$LibsFile" - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At C:\Users\thheller\Documents\WindowsPowerShell\Modules\ClojureTools\ClojureTools.psm1:353 char:5
+ & $JavaCmd @JvmCacheOpts @JvmOpts "-Dclojure.libfile=$LibsFile" - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ResourceUnavailable: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : NativeCommandFailed #2020-04-26 10:30:09 thheller windows powershell. I'm assuming that means the classpath is too long? #2020-04-26 10:52:26 dominicm I know people have reported long classpath issues on windows. #2020-04-26 10:52:45 dominicm Someone looked into copying whatever it is mvn does, I think it involved a file with the cp in. #2020-04-28 12:36:59 Bhougland total newb question, but where does deps.edn install the libraries? #2020-04-28 12:40:07 kszabo ~/.m2 and ~/.gitlibs #2020-04-28 12:40:10 kszabo by default #2020-04-28 12:40:46 teodorlu ~/.m2 is the same as Maven and Leiningen.
$ find ~/.m2 -iname 'clojure*jar'
/home/teodorlu/.m2/repository/net/mikera/clojure-utils/0.8.0/clojure-utils-0.8.0.jar
/home/teodorlu/.m2/repository/clojure-msgpack/clojure-msgpack/1.2.1/clojure-msgpack-1.2.1.jar
/home/teodorlu/.m2/repository/org/clojure/clojurescript/1.10.520/clojurescript-1.10.520.jar
/home/teodorlu/.m2/repository/org/clojure/clojurescript/1.10.597/clojurescript-1.10.597.jar
/home/teodorlu/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/home/teodorlu/.m2/repository/org/clojure/clojure/1.9.0/clojure-1.9.0.jar
/home/teodorlu/.m2/repository/clojure/java-time/clojure.java-time/0.3.2/clojure.java-time-0.3.2.jar
/home/teodorlu/.m2/repository/com/theoryinpractise/clojure-maven-plugin/1.7.1/clojure-maven-plugin-1.7.1.jar
/home/teodorlu/.m2/repository/com/cemerick/clojurescript.test/0.0.4/clojurescript.test-0.0.4.jar
#2020-04-28 12:40:46 Bhougland then the trace.edn tellls my project how to find them? #2020-04-28 12:42:30 Bhougland nevermind. that doesn't show a path on my local machine #2020-04-28 12:43:09 teodorlu Based on your deps.edn file, you get a classpath. Try running clj -Spath in a deps project, and you'll se it refer back into where your libraries are installed. #2020-04-28 12:44:19 Bhougland {:version "1.10.1.489"
:config-files ["/home/ben/.clojure/deps.edn" "deps.edn" ]
:config-user "/home/ben/.clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/nix/store/msn2kqpck9c5iin8hgjzm5chpwpm2167-clojure-1.10.1.489-prefix"
:config-dir "/home/ben/.clojure"
:cache-dir ".cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""} #2020-04-28 12:44:45 Bhougland Does this look okay? #2020-04-28 12:45:09 Bhougland I am questioning the :install-dir because I use Nixos and this path would be immutable #2020-04-28 12:48:16 alexmiller The install dir isn’t used anymore #2020-04-28 12:45:37 teodorlu I must admin that I'm not sure about what you're asking. #2020-04-28 12:46:05 Bhougland my repl can never mind my classpath #2020-04-28 12:47:26 teodorlu If you want to use deps.edn the "Nix way", I'm guessing you'll want guaranteed immutable references to what's normally a mutable folder in ~/.m2. But then you'll have to do some major modifications to tools-deps, I think.
Other people might be able to answer better. #2020-04-28 12:48:33 Bhougland I don't really want to do it the nix way. I just want to transistion to deps.edn and my repl yells at me that it can never find my classpath #2020-04-28 12:49:08 teodorlu Oh, so you've installed the clojure tool via Nix, it doesn't work, and you just want it to work? #2020-04-28 12:49:40 Bhougland Yes. But I am wondering if it is nix or my lack of understanding on some point #2020-04-28 12:52:18 Bhougland I appologize, earlier I posted Sdescribe not Spath. Spath is showing the .m2 directiory like it should #2020-04-28 12:53:53 Bhougland another vairable that is probably tripping me up is that I wired together Kakoune and the klipse-repl. I probably did something wrong there. #2020-04-28 12:54:17 Bhougland Thanks for the help #2020-04-28 12:56:08 teodorlu Don't think I'll be able to help you with that setup. If you want a simple experience getting started, the official Clojure installer + #calva should be a good place to start. #2020-04-28 12:57:04 Bhougland Thanks for the support, it helped me target the most likely issue. #2020-04-30 18:34:09 dominicm @alexmiller I know it's been discussed before, but one good reason to have access to the resulting deps.edn that built the current JVM is for file watching. I want to automatically configure figwheel to watch my :paths and my :local/root deps, but not my gitlibs (which are dirs on the classpath). Workaround is to filter anything containing ".gitlibs", but I'm not sure of the portability or future-proof of that (as soon as someone needs a config flag for that, the tower falls). #2020-04-30 18:35:12 alexmiller it's coming... #2020-04-30 18:36:07 alexmiller that work is done and in a branch but ties into ongoing work on add-lib and tools.build and maybe even some clojure core stuff #2020-04-30 18:39:21 alexmiller the new term we're using for this is the "runtime basis", which is a superset of the merged deps.edn that also includes your effective alias mods, the lib map and a (datafied) classpath. clj will compute this, cache the basis file, and inject a reference to it via a system property. we've got some additional apis to read and help you use the basis that may end up in clojure proper (the reading part does not depend on tools.deps - it's just data), but those are really just helpful, it's just reading edn and using clojure data #2020-04-30 18:40:21 alexmiller fyi, there is an env var that defines the gitlibs root, so you could base it on that (GITLIBS) #2020-04-30 18:40:30 dominicm Ah, interesting. Didn't know that. #2020-04-30 18:40:54 dominicm I've also just realized that the libfile is sufficient. I can figure out the intersection of classpath dirs & gitlibs from that. #2020-04-30 18:41:22 alexmiller so, that is currently being injected by clj as I describe above, but that's probably going to be replaced #2020-04-30 18:41:58 dominicm I've spent a lot of time this afternoon due to a collision of me trying to remove code that filtered libs out (but indirectly - so I didn't know that was the difference) & a bug figwheel where it can't load certain namespaces that are in watch dirs, and it's caused by a particular dependency we have... and now it's almost 20:00! #2020-04-30 18:42:00 alexmiller there is a system property clojure.libfile that points to the libfile used to start the current process #2020-04-30 18:42:43 alexmiller assuming you start with clj of course #2020-04-30 18:42:49 dominicm Hmm, okay. I'll probably make sure my code explicitly removes the bad dependency then, that way it'll continue working after the clj version bump. #2020-05-01 22:43:56 drewverlee whats the difference between
-Jopt Pass opt through in java_opts, ex: -J-Xmx512m
-Oalias... Concatenated jvm option aliases, ex: -O:mem
the first is jsut for one option and the later is for multiple? #2020-05-01 22:55:10 hiredman aliases are a feature of tools.deps, where you can kind of group alternative configurations together and give them a name, and -O lets you apply them #2020-05-01 22:55:37 hiredman -J is for passing arguments directly to the command that launches the jvm, typically 'java' #2020-05-01 22:56:32 drewverlee What would it look like to psas multiple arguments to the jvm? each one gets a J-<arg> #2020-05-01 22:56:45 drewverlee clojure J-<arg> J-<arg> #2020-05-01 22:56:56 hiredman passing arguments is bad wording on my part #2020-05-01 22:57:17 hiredman it is passing options, flags like -Xmx512m #2020-05-01 22:58:12 hiredman so for example, you could pass -J-server (which is the default these days) #2020-05-01 22:58:36 drewverlee so for multiple its like clojure ... -J-Xmx512m -j-Xms512 #2020-05-01 22:58:53 seancorfield (! 613)-> clj -J-Dfoo=bar -J-Dquux=wibble
Clojure 1.10.1
user=> (System/getProperty "foo")
"bar"
user=> (System/getProperty "quux")
"wibble"
user=> ^D
#2020-05-01 22:59:18 hiredman I would put them first #2020-05-01 22:59:36 seancorfield One -J per argument you want to pass. Beware of white space 🙂 #2020-05-01 22:59:53 hiredman when you run clojure or clj the argument handling is complicated #2020-05-01 22:59:59 drewverlee nice. i didnt know about system/getProperty. #2020-05-01 23:00:28 hiredman some arguments are for scripts, some for jvms, and some for clojure, and some for your code #2020-05-01 23:02:44 hiredman clj/clojure also have another of different ways to combine aliases, but -A will usually do want you want #2020-05-01 23:03:45 hiredman (aliases can also specify jvm options, and -O combines those ignoring the other stuff specified in the aliases) #2020-05-02 19:50:38 seancorfield The installer's source is here https://github.com/clojure/brew-install (the shell scripts etc) and that readme talks about the components, including the uberjar above @stebokas #2020-05-03 10:03:42 victorb Hello! I've got an idea for my own extension to tools.deps.alpha, I can see bunch of them in the src/main/clojure/clojure/tools/deps/alpha/extensions/ path. So let's say I now have my X extension written, how can I make tools.deps.alpha use it? I guess I want to end up with something similar to a lein plugin where installing it gives you the ability of using the new extension. I'm not super familiar with Java (mostly been dealing with ClojureScript) so don't expect anyone to give me the straight path to my solution, but some pointers/keywords for what to lookup would be most appreciated! #2020-05-03 10:07:29 dominicm Using it with the library is easy, using it with the cli is not yet possible #2020-05-03 10:10:17 victorb I see. Thanks! So for now, if I'd like to publish it to enable others to use it, I would have to publish a forked version of tools.deps.alpha with my extension worked into the codebase? #2020-05-03 10:15:29 dominicm No. They would just require your namespace which extends the multi method #2020-05-03 10:18:51 victorb Hm, but if this is not exposed to do in the cli, it feels like a workaround at best, only for people who are using deps as a library. Not sure how the common use case looks for using deps (as I'm still mostly using leiningen), but feels like most people would still stick to using the cli. But again, I don't know enough about this to say for sure #2020-05-03 10:32:39 dominicm In that case you'll need to fork the cli #2020-05-03 10:34:32 victorb alright. Thanks a lot for the explanations! #2020-05-03 12:16:17 alexmiller What’s your extension? #2020-05-03 12:25:50 victorb @U064X3EF3 fetching dependencies via a content-addressed database (IPFS or similar) and splitting versions into two different versions, one for the interface and one for the implementation #2020-05-03 12:32:19 alexmiller Including an extension would just require changing the one line in the clojure script that defined the tools classpath #2020-05-03 12:33:02 alexmiller Well, I guess the extension also needs to be loaded #2020-05-03 12:35:06 alexmiller clj has been out for 2 or 3 years and this is the first time someone has asked about this so it hasn’t really been a priority #2020-05-03 12:50:56 victorb yeah, if I'll fork tools.deps I can see the path forward easily, but for obvious reasons I wanted to avoid that.
Makes sense that no one asked, as it's a bit bleeding-edge stuff to load dependencies from something that is not HTTP/S
Thanks Alex! #2020-05-03 13:05:27 dominicm I think it gets more complex if you want to support dependencies which depend on an extension. #2020-05-03 14:22:27 victorb Hm, yeah, might have to have some sort of bootstrapping available or ship whatever the extension comes with over clojars/maven. Unless I just end up with a external babashka cli util that does the fetching... Not sure yet, exploring the options #2020-05-03 16:58:55 alexmiller Figuring all that out is why we haven’t done it yet :) #2020-05-03 15:44:31 murtaza52 what is the lein clean equivalent for cli tools ? #2020-05-03 15:57:34 dominicm clj -Sforce #2020-05-03 17:03:29 alexmiller There isn’t anything to clean #2020-05-03 17:04:09 alexmiller Force will force a recompute of the cached classpath which is needed in a few special cases but rare #2020-05-03 17:05:14 alexmiller In lein, there is a build directory with artifacts, but that’s not a thing in clj (YET!) #2020-05-03 17:08:14 dominicm Does that mean it will be a thing in clj, or that it will be a thing in a tools.build main? #2020-05-03 17:08:27 dominicm (wondering how integrated these things are) #2020-05-03 17:10:00 vlaaad I hope tools.build will have some sha hashes or something for its inputs/outputs caching so it won't require cleaning... #2020-05-03 17:14:09 alexmiller Not going to get into any details now, sorry :) #2020-05-03 17:16:17 dominicm We'll have to learn how to bribe you. I hear the ops team at facebook has a full wall of alcohol. #2020-05-04 12:21:53 tianshu How can I make clojure cli download jars if Clojure 1.10.1 in ~/.m2 is not found. #2020-05-04 12:27:52 alexmiller try using -Sforce #2020-05-04 12:28:10 alexmiller that will force the classpath to be recomputed, not used from cache and should trigger a download #2020-05-04 12:31:24 tianshu @alexmiller Thanks! it works #2020-05-04 14:59:19 drewverlee Is there a way to run the clj/clojure script so it doesn't try to resolve deps first? were getting a classpath error and were hoping to get some diagnostic information via J-XshowSettings:properties, but we hit the "cannot find artifact" error first. #2020-05-04 15:07:17 alexmiller if you don't resolve deps, you can't make a classpath, and thus can't start a JVM to use that setting, so no? #2020-05-04 15:09:53 drewverlee That makes sense. I guess i need to oreint myself, clj assumes you can start a JVM, but for debugging their is some information you can get before that. So i can just run java -XshowSettings:properties. #2020-05-04 15:10:32 alexmiller is that a thing? #2020-05-04 15:11:09 drewverlee is what a thing? 🙂 #2020-05-04 15:11:40 alexmiller showsettings #2020-05-04 15:11:53 alexmiller I see it is - was confused by the help message after nvm #2020-05-04 16:25:36 drewverlee If a deps procurer (local/root) points to another deps with a Procurer (e.g mvn/repos) Can i assume that it will read that Procurer or does it just grab the deps? #2020-05-04 16:35:17 alexmiller if you're asking whether transitive dependency maven repos will be used to resolve dependencies, then no #2020-05-04 16:35:58 alexmiller maven allows this, but it seems awfully dicey from a security pov so I kind of consider it an anti-feature #2020-05-04 16:46:01 drewverlee its likely i'm not describing this correctly.
I have a deps.edn with {:deps {datomic {:local/root "../datomic"}} and in "../datomic" the deps reads {:mvn/repos {"http://my.datomic.com" {:url "https..."}} It seems i have to add the mvn repos to the deps with the local/root procurer, even though that deps doesn't directly have a dep on datomic-pro. It has that dependency through the other deps.
@alexmiller as always, thanks for the help. #2020-05-04 18:00:38 alexmiller Yeah, that’s the scenario I’m describing #2020-05-04 18:01:10 alexmiller The question is, do you want your transitive deps to make decisions about where you get things on the internet? #2020-05-04 18:02:10 alexmiller The answer is definitely not an unqualified yes #2020-05-04 18:02:19 alexmiller So it needs a lot of thought #2020-05-04 18:10:06 drewverlee i understand. thanks alex. #2020-05-04 20:05:34 vlaaad okay, I know how to bait Alex Miller to tell us some more about tools-build, watch this: #2020-05-04 20:06:02 vlaaad tools-build is a fork of scala's SBT #2020-05-04 20:27:25 alexmiller do you want me to write about it, or do you want me to finish it? :) #2020-05-04 20:43:00 dominicm I was going to make a joke reply, but now I'm trying to decide my actual stance on it... If I know the high level goal I can plan around it and start getting my chickens in a row 🐔 #2020-05-04 21:18:33 seancorfield @dominicm I'm curious how your chickens would be organized differently based on various possible high level goals of tools.build? #2020-05-05 07:35:52 dominicm Apparently my ducks would be because I was a bit daft last night #2020-05-05 07:45:24 dominicm Anyway. I usually find libraries from core require a shift in thinking or approach. Eg clj forces you to do your own builds, so I'd expect the ecosystem to start building -mains earlier in anticipation.
I have done my best to make sure my build functions can be used in different ways, but I'm betting I missed something.
Knowing whether it solves the java sources problem would allow people using lein currently who want to move off to start doing any prep they need, as I doubt there will be something as straightforward as :java-sources. #2020-05-05 15:09:17 seancorfield I'm not sure what "prep" those lein users could realistically do, even if the goal was stated as "solves the java sources problem"? #2020-05-05 15:13:39 seancorfield If tools.build can completely replace depstar, I'll be happy to switch to it. That means: build JAR and uberJAR, do AOT, specify the main namespace, Multi-Release JAR support, merging data readers (and several other types of files) in appropriate ways... If it also includes Java compilation, I might change how we build/deploy a couple of OSS Java projects that we currently build from source (manually, as needed) and use local JAR files for. #2020-05-05 06:56:16 murtaza52 there was a strange bug, I had deleted my .m2 directory some time back, and when I started my nrepl, it would start, but then on compiling a file it would give an error of a missing dep.
I changed the deps.edn file, and this triggered something, and on next nrepl startup, it downloaded the dependency. have never seen such a behaviour before. #2020-05-05 13:25:40 alexmiller that all is what I would expect #2020-05-05 13:26:07 alexmiller classpath is computed and then in cached in your .cpcache directory, with pointers into your .m2 #2020-05-05 13:27:07 alexmiller clj script doesn't re-check that those files still exist (it could do so, but there are some performance implications and it's a little messy) #2020-05-05 13:28:03 alexmiller changing the deps.edn makes the cached cp stale (it checks that the cp file is newer than all the source deps.edn files) #2020-05-05 13:29:43 alexmiller if you get into this situation again, use -Sforce to force the cp to be recomputed, which will also trigger a redownload #2020-05-05 17:35:49 rogerio haruo adachi Hey, I'm running into some issues here, regarding transitive dependencies with deps with logging dependencies (cambium + unilog).
Scenario is as follows:
1. I have a shared library that has common libs ( datomic in memory, datomic client, kafka, integrant ... and logging)
2. An application that imports the above shared library
I have made all the necessary exclusions in item 1.
But when I import in item 2, I got conflicts between slf4j many libraries that I have excluded in item 1.
How can I maintain only dependencies required (with exclusions) in item 1 in a transitive way? #2020-05-05 18:21:56 alexmiller can you share your dep.edn to repro? #2020-05-05 19:02:47 rogerio haruo adachi Sure. #2020-05-05 19:04:43 rogerio haruo adachi this is my libs deps
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
fundingcircle/jackdaw {:mvn/version "0.6.9"
:exclusions [org.slf4j/slf4j-log4j12
commons-logging/commons-logging
log4j/log4j
org.slf4j/slf4j-nop
org.slf4j/log4j-over-slf4j
org.clojure/tools.logging]}
datascript/datascript {:mvn/version "0.17.1"}
willa {:mvn/version "0.1.2"}
com.cognitect/anomalies {:mvn/version "0.1.12"}
com.gfredericks/test.chuck {:mvn/version "0.2.10"}
org.apache.kafka/kafka-streams {:mvn/version "2.3.0"
:exclusions [org.slf4j/slf4j-api]}
org.apache.kafka/kafka-streams-test-utils {:mvn/version "2.3.0"}
io.pedestal/pedestal.service {:mvn/version "0.5.7"}
io.pedestal/pedestal.route {:mvn/version "0.5.7"}
io.pedestal/pedestal.jetty {:mvn/version "0.5.7"}
integrant {:mvn/version "0.7.0"}
integrant/repl {:mvn/version "0.3.1"}
duct/core {:mvn/version "0.8.0"}
com.datomic/client-cloud {:mvn/version "0.8.78"
:exclusions [commons-logging/commons-logging]}
com.cognitect.aws/api {:mvn/version "0.8.408"
:exclusions [org.clojure/tools.logging]}
com.cognitect.aws/endpoints {:mvn/version "1.1.11.689"}
com.cognitect.aws/s3 {:mvn/version "780.2.583.0"}
rewrite-clj {:mvn/version "0.6.1"}
compute/datomic-client-memdb
{:git/url ""
:sha "fd880e70b45a0837f1604aa1230047cb74ec536e"
:exclusions [org.slf4j/slf4j-nop org.slf4j/log4j-over-slf4j org.slf4j/jcl-over-slf4j]}
badigeon/badigeon
{:git/url ""
:sha "dca97f9680a6ea204a2504c4414cafc4ba182a83"
:exclusions [org.slf4j/slf4j-nop
org.slf4j/jcl-over-slf4j]}
org.clojure/tools.cli {:mvn/version "0.4.2"}
cambium/cambium.core {:mvn/version "0.9.3"}
cambium/cambium.codec-simple {:mvn/version "0.9.3"}
spootnik/unilog {:mvn/version "0.7.24"}} #2020-05-05 19:06:55 rogerio haruo adachi And this is my app Deps
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
my.lib {:mvn/version "0.1.11-SNAPSHOT"} ;; This is where I import my lib and get conflicts over slf4j log4j libs
tick {:mvn/version "0.4.20-alpha"}
integrant/repl {:mvn/version "0.3.1"}
com.gfredericks/test.chuck {:mvn/version "0.2.10"}} #2020-05-05 19:56:28 alexmiller are you sure you're actually getting the latest version of your snapshot lib? by default, maven will only refresh snapshots once per day #2020-05-05 22:37:31 rogerio haruo adachi Yes, I have been working with the latest version of my lib (snapshot) #2020-05-05 20:13:37 alexmiller what conflicts are you seeing? #2020-05-05 20:14:54 alexmiller I see log4j-over-slf4j under spootnik/unilog and other slf4j stuff under cambium - you should look at clj -Stree and see if what you're getting makes sense #2020-05-05 20:16:51 alexmiller if you want a more fine-grained analysis, use clj -Strace - that will generate a trace.edn file and you can walk through the :log to see each library it considers in order, whether it included it or not, and why #2020-05-05 20:19:27 alexmiller as a fallback, including it as an explicit dep in your app deps will take precedence over anything below (none of those exclusions are necessary then) #2020-05-05 22:38:49 rogerio haruo adachi Thanks @alexmiller the fallback choice might work for now. Mean while I have extracted logging dependencies into a separate library to avoid conflicts.
Thanks for your help #2020-05-06 15:16:46 ikitommi After removing a directory from my .m2/repository, my clj failed the find dependencies using clj -m nrepl.cmdline -i -C. After doing any modification to the deps.edn , it reloaded the missing libs from clojars. Is there some local cache that caused this? #2020-05-06 15:17:38 ikitommi e.g. “`deps.edn` unchanged, we have all the deps already”? #2020-05-06 15:20:36 delaguardo class path cache is in .cpcache
and yes, unchanged deps.edn means all dependencies already downloaded #2020-05-06 15:47:35 ikitommi Thanks, I should have read the history better. #2020-05-06 16:49:50 seancorfield clj -Sforce is your friend here. I've tripped over this too. #2020-05-06 19:57:24 kenny We have been getting this exception fairly frequently on CI builds. Retrying the job succeeds. Any idea what this is?
Error building classpath. class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
java.lang.ClassCastException: class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1882)
at java.base/java.util.HashMap$TreeNode.treeify(HashMap.java:1998)
at java.base/java.util.HashMap.treeifyBin(HashMap.java:767)
at java.base/java.util.HashMap.putVal(HashMap.java:639)
at java.base/java.util.HashMap.put(HashMap.java:607)
at java.base/java.util.HashSet.add(HashSet.java:220)
at org.apache.maven.model.validation.DefaultModelValidator.validateId(DefaultModelValidator.java:847)
at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependency(DefaultModelValidator.java:659)
at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependencies(DefaultModelValidator.java:583)
at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveModel(DefaultModelValidator.java:373)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:494)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
at org.apache.maven.model.building.DefaultModelBuilder.importDependencyManagement(DefaultModelBuilder.java:1299)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:474)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:292)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
at clojure.tools.deps.alpha.extensions.maven$eval813$fn__815.invoke(maven.clj:99)
at clojure.lang.MultiFn.invoke(MultiFn.java:244)
at clojure.tools.deps.alpha$expand_deps$fn__1230$fn__1231.invoke(alpha.clj:214)
at clojure.tools.deps.alpha.util.concurrent$submit_task$task__500.invoke(concurrent.clj:33)
at clojure.lang.AFn.call(AFn.java:18)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
#2020-05-06 20:01:25 hiredman that is a race condition when using a hash map from multiple threads, there was a recent-ish change to clj to do some stuff in parallel, and I bet if you disabled that those exceptions would go away #2020-05-06 20:04:12 hiredman you can stick :threads 1 in your deps.edn somewhere to download single threaded #2020-05-06 20:11:59 alexmiller No it’s a clj flag #2020-05-06 20:12:08 alexmiller -Sthreads 1 #2020-05-06 20:12:52 alexmiller (But going to be a resolve-args alias) #2020-05-06 20:13:34 alexmiller I haven’t seen this error anywhere before #2020-05-06 20:16:35 alexmiller I will file a ticket to take a closer look #2020-05-09 16:31:33 dpsutton i'm trying to get a repl up and running in the tools.deps repo and running into a class not found exception for clojure.tools.deps.alpha.util.S3TransporterFactory. This seems quite reasonable but I don't see any instructions on how to build the java portion of this project. Does anyone know what incantation i need to provide to have this class on the classpath? #2020-05-09 17:20:27 alexmiller the project is built with mvn package #2020-05-09 17:26:10 seancorfield All Contrib libraries are, yes? #2020-05-09 17:26:55 alexmiller yes #2020-05-09 17:29:12 alexmiller you can use mvn to get a repl too with mvn clojure:repl but it's a pretty crappy repl #2020-05-09 17:30:03 alexmiller I typically use it as a maven project in Cursive #2020-05-09 17:30:41 dpsutton ah. i'm learning cursive and reflexively imported it as a deps project. i'll figure out how to switch that #2020-05-09 17:31:26 dpsutton thanks much #2020-05-12 11:47:34 plexus when you have a dependency with a :local/root to another clojure-cli/deps.edn project, is there any way to have it pick up aliases in the deps.edn that is being referenced? #2020-05-12 11:48:04 plexus I thought they might "inherit" whatever aliases are in use, but seems that's not the case. #2020-05-12 12:25:48 alexmiller Currently, no #2020-05-12 12:27:57 alexmiller What’s the actual problem you’re trying to solve? #2020-05-12 16:12:14 holyjak Where does deps download dependencies to (ie which folder to back up for caching on CI)? ~/.m2? #2020-05-12 16:15:35 ghadi .m2 for maven, .gitlibs for git dependencies #2020-05-12 16:43:55 alexmiller #2020-05-12 17:12:21 ghadi in Github Actions I use:
- name: Cache Maven
uses: actions/ #2020-05-12 17:12:24 ghadi to handle caching #2020-05-12 17:53:47 avi 👋 I feel like this might be a FAQ but I did search the mirror in Zulip to no avail, so:
I have a new project that a teammate is trying to work on. (They’re new/rusty to/with Clojure.)
All the Git deps in the deps.edn are specified with https URLs.
For some reason I don’t understand, when my teammate tries to start a REPL, and thereby triggers the retrieval of the deps, they’re getting this error:
Error building classpath. @github.com/cognitect-labs/anomalies: invalid privatekey: [
This is confusing because this dep has this coord:
{:git/url ""
:sha "2decbbb3ffd8e919c67d29f39ec6d920575f65c3"
:tag "0.1.12"
:deps/manifest :deps}
Full deps.edn: https://github.com/FundingCircle/md2c8e/blob/master/deps.edn
Is this a known problem with a known solution? Am I just missing something?
Thanks! #2020-05-12 18:03:53 alexmiller that's weird looking #2020-05-12 18:04:04 alexmiller I cloned that repo and was able to start a repl #2020-05-12 18:04:41 avi Yeah, I’ve been wracking my brain to try to think of something I did wrong here… maybe my teammate’s git config/environment is all wonky? #2020-05-12 18:05:00 alexmiller well, deps mostly doesn't use your git config at all #2020-05-12 18:05:48 alexmiller I would definitely re-try with -Sforce just to make sure you don't have a bad cached classpath #2020-05-12 18:06:09 alexmiller and look at ~/.m2/settings.xml to see if there are any funky repos set up #2020-05-12 18:06:32 alexmiller and I guess you should look at ~/.clojure/deps.edn in case there is anything in there #2020-05-12 18:07:53 alexmiller it may be that the error is coming from the guts of jsch under jgit and the url has been translated so that being different than what's in deps.edn is not the issue #2020-05-12 18:09:22 avi ok, thank you! #2020-05-12 18:29:04 alexmiller might be something weird seen by the ssh-agent that's totally unrelated, dunno #2020-05-12 20:38:49 johanatan hi, is it possible to have a deps.edn alias to shell out to normal bash commands ? #2020-05-12 20:39:06 johanatan we have a preprocessing step that needs to run before figwheel-main is started (for example) #2020-05-12 20:39:27 johanatan and i'd like to do: clj -A:preprocess:fig:repl or clj -A:preprocess:fig:build #2020-05-12 20:39:44 alexmiller Yes, depending on how much pain you’re willing to endure #2020-05-12 20:39:58 johanatan hehe, what are my poisons? #2020-05-12 20:40:05 alexmiller Or maybe not as a single command #2020-05-12 20:40:27 alexmiller A single clj command line will run one Clojure program #2020-05-12 20:40:42 alexmiller Clojure can shell our and run anything #2020-05-12 20:41:00 johanatan yea, sounds fine. so you're saying to use -m the-main and shell out from the-main ? #2020-05-12 20:42:03 alexmiller Yeah - either using clojure.shell or actually I find doing the interop with Java’s ProcessBuilder to be pretty trivial #2020-05-12 20:42:26 alexmiller But if you’re running two commands, why not just write bash or Babashka or something #2020-05-12 20:42:51 johanatan it's just one command really: npm run build to build our webpack #2020-05-12 20:54:19 johanatan this did the trick:
(ns my-ns.webpack
(:require
[clojure.java.shell]))
(defn -main []
(let [{:keys [exit out err]}
(clojure.java.shell/sh "npm" "run" "build" :dir "./webpack")]
(assert (= exit 0))
(println out)
(System/exit 0)))
#2020-05-12 20:59:59 ghadi what's the point #2020-05-12 21:00:19 ghadi @johanatan what do you gain doing this in clojure? #2020-05-12 21:02:30 johanatan i gain the ability to do: clj -A:preprocess:fig:repl or clj -A:preprocess:fig:build . it's a single command for our engineers to enter rather than multiple . easier to document/ transfer. and it still allows clj to be the top-level entry point rather than a bash or other script which would have to be maintained if our deps.edn setup changes. this way, if deps.edn setup changes, it's merely the addition of another "-A" #2020-05-12 21:02:53 johanatan it's a "turnkey" solution. one command startup type situation #2020-05-12 21:05:11 johanatan although there seems to be a catch, the repl 's :main-opts is apparently overriding the preprocess one (which effectively turns preprocess into a no-op). #2020-05-12 21:09:59 alexmiller yes, that's my point above about an invocation of clj running only one program - you need to do two things here #2020-05-12 21:06:22 johanatan @borkdude yea, i saw that and explored it. but it requires two additional installs (at least): deps.clj and babashka. #2020-05-12 21:06:36 johanatan and since this will be running in GitHub Actions, i wanted to minimize installs #2020-05-12 21:06:53 borkdude @johanatan deps.clj doesn't require babashka, it's also available as a standalone executable #2020-05-12 21:06:57 borkdude but point taken #2020-05-12 21:07:00 johanatan it would be nice if there could be a self-install of these tools #2020-05-12 21:07:06 johanatan like say cljfmt-runner.check #2020-05-12 21:07:12 johanatan directly within the deps.edn #2020-05-12 21:07:46 johanatan like this i mean:
:lint {:extra-deps {com.jameslaverack/cljfmt-runner
{:git/url ""
:sha "97960e9a6464935534b5a6bab529e063d0027128"}}
:main-opts ["-m" "cljfmt-runner.check"]}
#2020-05-12 21:08:16 borkdude @johanatan deps.clj is available as a standalone executable. in CI you would only have to install that instead of clojure, so one thing instead of another thing, not an extra thing #2020-05-12 21:08:55 borkdude and "installing" is just downloading that executable to the path, that's it #2020-05-12 21:10:07 johanatan yea but there is a github action for tools.deps:
uses: vouch-opensource/
unfortunately only one uses per step. but yea i could do a manual install #2020-05-12 21:11:15 borkdude just giving you the option, I'm not saying you should 🙂 - running npm using a clojure command, feels like a detour to me #2020-05-12 21:11:42 johanatan hehe, true. i'll think about it. thanks! #2020-05-12 21:12:21 johanatan (for those tools) #2020-05-12 21:13:03 borkdude that link doesn't work for me #2020-05-12 21:13:20 johanatan :lint {:extra-deps {com.jameslaverack/cljfmt-runner
{:git/url ""
:sha "97960e9a6464935534b5a6bab529e063d0027128"}}
:main-opts ["-m" "cljfmt-runner.check"]}
#2020-05-12 21:13:46 johanatan ^^ including babashka as a git-based "extra-dep". i guess it would be impossible for deps.clj to be used that way #2020-05-12 21:15:08 borkdude deps.clj doesn't install anything, it's a drop-in replacement for clojure that has some benefits for some people (like Windows users that want to invoke it in cmd.exe). And some custom options. Nothing more than that. #2020-05-12 21:16:06 borkdude of course you can script an installer in some Clojure file and put that behind a main-opts #2020-05-12 21:16:17 johanatan ah right. good point i guess it wouldn't require babashka but for this simple script, babashka would perhaps be a better option than fullblown clojure #2020-05-12 21:17:02 johanatan or.. i guess you're saying no need for either since i could invoke npm directly #2020-05-12 21:17:10 borkdude yeah #2020-05-12 21:18:05 johanatan ah i see #2020-05-12 21:20:05 johanatan alright you've convinced me. deps.clj it is #2020-05-12 21:20:26 johanatan developers will just install it instead of clojure #2020-05-12 21:20:37 johanatan (or both if they so like) #2020-05-12 21:22:48 johanatan but can the argument to -S be placed inside deps.edn ? #2020-05-12 21:23:12 johanatan (as an alias) #2020-05-12 21:23:30 borkdude no, deps.clj doesn't change anything to deps.edn #2020-05-12 21:24:39 johanatan hmm, yea, not sure that helps in this case as it would mean two separate invocations of deps.clj #2020-05-12 21:25:00 johanatan probably the "script" feature of package.json itself is the best bet since those can be strung together / combined #2020-05-12 21:25:13 borkdude yeah, if you're already using npm I'd use that #2020-05-12 21:25:19 johanatan :+1: #2020-05-12 21:33:48 ghadi vouch-opensource/tools.deps-build
I do not recommend that @johanatan. I have found a lot of success with DeLaGuardo/ in Github actions #2020-05-12 21:34:06 ghadi it allows you to run clj as many times as you need #2020-05-12 21:34:21 ghadi vouch-opensource/tools.deps-build makes you cram it all into one invocation #2020-05-12 21:43:10 johanatan Oh yes, haha, that cram all into one was part of the motivation here as well. I’ll look at making the change to setup-clojure. Thx! #2020-05-12 21:44:04 ghadi - name: Setup Java
uses: actions/ #2020-05-12 21:44:09 ghadi then I go buck wild #2020-05-12 21:46:24 johanatan :+1::skin-tone-2: #2020-05-12 21:46:40 johanatan If you needed NPM, you’d just install it in a prior step? #2020-05-12 21:46:55 ghadi npm is installed by default in Actions, IIRC #2020-05-12 21:47:03 ghadi already available #2020-05-12 21:47:05 johanatan Ah cool. Yea I thought so too #2020-05-14 00:18:28 fabrao hello all, is there any way to exclude dependency of a lib in deps.edn ? #2020-05-14 00:20:12 alexmiller you can use :exclusions [lib/name] in a coordinate #2020-05-14 00:21:00 alexmiller I'm assuming you're talking about a transitive dependency #2020-05-14 00:21:10 alexmiller there is no global exclusions right now #2020-05-14 00:22:36 fabrao Hello Alex, I´m doing this
{:paths ["src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
mount {:mvn/version "0.1.16"}
com.sikulix/sikulixapi {:mvn/version "2.0.4"
:exclusions [org.bouncycastle/bctsp-jdk14]}}}
and still getting
clj -J'-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl}'
Error building classpath. Could not find artifact bouncycastle:bctsp-jdk14:jar:138 in central () #2020-05-14 00:23:44 fabrao what is maybe wrong? #2020-05-14 00:24:15 alexmiller not sure, that seems like good deps syntax #2020-05-14 00:25:17 alexmiller I can repro locally #2020-05-14 00:26:19 alexmiller that 138 looks suspicious to me #2020-05-14 00:26:32 alexmiller maybe should be 1.38 #2020-05-14 00:26:49 fabrao so, excluding is not an option for this? #2020-05-14 00:26:51 alexmiller which is an actual version - I assume that's in com.sikulix/sikulixapi deps #2020-05-14 00:27:11 alexmiller might be an edge case where it's failing while getting the metadata for something that it's going to exclude #2020-05-14 00:27:33 alexmiller I am getting on a call, but I can look at it tomorrow #2020-05-14 00:27:57 fabrao oh, ok, thanks #2020-05-14 00:28:13 fabrao I´ll try with lein #2020-05-14 00:31:04 seancorfield @fabrao apparently what you want to exclude is bouncycastle/bctsp-jdk14 (without the org.) #2020-05-14 00:32:20 seancorfield I noticed in the initial set of download messages:
Downloading: org/jboss/logging/jboss-logging/3.1.4.GA/jboss-logging-3.1.4.GA.pom from central
Downloading: org/jboss/jboss-parent/9/jboss-parent-9.pom from central
Downloading: bouncycastle/bctsp-jdk14/138/bctsp-jdk14-138.pom from central <======== LOOK AT THIS LINE!
Downloading: org/bouncycastle/bctsp-jdk14/1.38/bctsp-jdk14-1.38.pom from central
Downloading: org/ghost4j/ghost4j/1.0.1/ghost4j-1.0.1.jar from central
#2020-05-14 00:32:48 fabrao @seancorfield You are the BOSS !!! That´s it !!! Thank you #2020-05-14 00:33:21 seancorfield (which looks ancient and maybe even bogus?) #2020-05-14 00:55:42 alexmiller yeah, that must be old or something #2020-05-14 00:59:05 alexmiller in case you're wondering the path to badness is
com.sikulix/sikulixapi 2.0.4
net.sourceforge.tess4j/tess4j 4.4.1
org.ghost4j/ghost4j 1.0.1
com.lowagie/itext 2.1.7
bouncycastle/bcprov-jdk14 138 #2020-05-14 00:59:31 alexmiller there are newer versions of stuff lower down, didn't investigate in detail but probably could update some stuff to newer there to fix #2020-05-14 01:17:07 fabrao I could use with excludes like Sean said, thanks #2020-05-14 07:37:35 Andreas Edvardsson I have added a plain java class alongside my clojure code. Is it possible to have it compiled as a part of the ordinary clojure compilation? In lein there seems to be a :java-source-paths option, is there something corresponding in tools.deps? #2020-05-14 07:46:04 plexus not in tools.deps. Some of the build tools based on tools.deps can do it though. Or you may be able to get away with compiling them manually, e.g.
for x in **/*.java; do javac -cp `clojure -Spath` $x ; done
#2020-05-14 08:35:24 Andreas Edvardsson Thanks for the answer! The class is expected to be more or less static, so just a manual javac invocation indeed seems to be the simplest solution. :) #2020-05-14 22:04:00 bringe Hi, I'm a bit new to the clojure command line tools. I'm wondering if there is an equivalent to lein deps and this is because I'm using it with docker. It is recommended with docker to download dependencies in a separate step so that they aren't re-downloaded if none changed, such as:
FROM clojure
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY project.clj /usr/src/app/
RUN lein deps
COPY . /usr/src/app
RUN mv "$(lein uberjar | sed -n 's/^Created \(.*standalone\.jar\)/\1/p')" app-standalone.jar
CMD ["java", "-jar", "app-standalone.jar"]
#2020-05-14 22:04:33 alexmiller you can do clj -e nil to trigger deps to resolve and download #2020-05-14 22:04:44 bringe Ahhh thanks! #2020-05-14 22:11:03 pmonks Yeah I had a need for this recently (in a TravisCI build), and ended up doing clojure -Srepro -e '(println "Dependencies downloaded")'. Felt a bit hackish, but glad to see that’s basically how it’s done. #2020-05-14 22:12:05 pmonks I stand corrected - it was a GitHub action/workflow thingy - but basically the same idea. #2020-05-16 15:01:17 gfredericks can a deps.edn provide jvm-opts for default calls, without having to specify an alias? #2020-05-16 15:01:52 alexmiller No #2020-05-16 15:02:08 gfredericks is that intentional, or a maybe-future-feature? #2020-05-16 15:02:20 alexmiller Maybe future #2020-05-16 15:02:50 alexmiller But it seems weird that all clj invocations in a project would share those #2020-05-16 15:03:03 alexmiller Which is why I haven’t added it yet #2020-05-16 15:03:37 gfredericks when imagining it was intentional I started thinking that maybe for a nontrivial project clojure isn't meant to be called directly, and you'd setup bin/ scripts that call clojure with appropriate aliases #2020-05-16 15:04:06 gfredericks at least for running specific code; maybe it's fine for launching a repl #2020-05-16 15:05:15 alexmiller Like maybe it would make more sense if it was scoped only to running a repl #2020-05-16 15:05:42 alexmiller Or running a specific main #2020-05-16 15:06:22 gfredericks yeah, specific main would definitely be adequate for anything I want to do #2020-05-16 15:12:40 alexmiller But if you’re doing that, don’t you already have an alias where you can put it? #2020-05-16 15:13:26 alexmiller Or is it that you don’t want to repeat it in aliases #2020-05-16 15:17:09 gfredericks hmm, you're right, I guess a human typing in a main ns might not be very common
so my particular situation, which is maybe not common enough to be worth targeting, is that my "project" mostly consists of a monolithic clj file at the root, and I just run clojure code.clj a lot
but I'd like to have it with a restricted -Xmx by default, and can't do that; I just have to remember to run clojure -Afoo code.clj instead #2020-05-16 15:17:55 gfredericks I think "stop being weird" is a perfectly good answer here, I was just curious #2020-05-16 19:11:15 seancorfield Or even just clojure -A:foo since you could add code.clj to the alias (as a :main-opts element), right? #2020-05-16 19:16:51 seancorfield {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "bac9dfdbd4d9fafeffe9f1eef5ea97ebef88eff08bf4"}, :content ("[email protected]")} #2020-05-16 21:30:12 dominicm If tools.deps exposed a :init-opts option in deps.edn, that would remove the need for multiple mains. Most uses (I've had) involve side-effects like launching an nrepl for personal dev purposes. If I use a main for that, I can't use a project-specific main. #2020-05-16 21:56:35 seancorfield :main-opts can contain -e and -i -- and you can combine multiple :main-opts from multiple aliases @dominicm #2020-05-16 21:56:41 seancorfield That's partly why I separate out :test and :runner for example. #2020-05-16 21:58:08 seancorfield I admit, I do miss the ability of combining aliases in other aliases -- that's probably the thing that would add the most ergonomic utility for me. That's really why we have a shell script wrapped around clojure: to do some "alias expansion" #2020-05-16 21:58:42 dominicm @seancorfield you can only have one :main-opts active from an alias at a time. #2020-05-16 21:59:26 dominicm Compare with jvm-opts which does concatenate. #2020-05-16 22:00:59 seancorfield Ugh! Yeah, you're right. I'd convinced myself they combine... #2020-05-16 22:06:29 seancorfield Which means I change I made recently to my dot-clojure file doesn't actually work. #2020-05-16 22:09:32 seancorfield (well, it does work, but not quite how I thought and it explains why I still need a manual step at the end) #2020-05-16 22:10:29 seancorfield So, yeah, you've now convinced me that having an :init-opts for just "init opts" that actually combined like :jvm-opts would be a good thing 🙂 #2020-05-16 22:10:58 seancorfield (or, frankly, even if :main-opts itself combined and it was just "caveat programmer" to use it that way) #2020-05-16 22:14:54 dominicm The problem with main opts composing is that there's an order problem. Not so with init-opts.
I suppose people may also be relying on the fact that main-opts conflict. #2020-05-16 22:15:29 seancorfield If you have multi -e options you might want them to execute in a particular order 🙂 #2020-05-16 22:16:21 seancorfield I added -e to my :deps alias to set up a DCL, so that if you do -A:deps:socket then you get a DCL in place before the Socket REPL is created so you able to use add-lib via Atom. #2020-05-16 22:16:44 seancorfield But if you do -A:deps:socket:rebl-11 you get a REBL window without DCL 😞 #2020-05-16 22:18:11 seancorfield So you have to do -A:deps:socket -R:rebl-11 -r (omitting the :main-opts from REBL) and that sets up a DCL and starts a Socket REPL and finally starts a command-line REPL... and in there you can do ((requiring-resolve 'cognitect.rebl/ui)) to start the REBL window and then it does have a DCL. #2020-05-16 22:18:28 seancorfield (this is all a workaround because Socket REPLs don't use DCLs by default) #2020-05-16 22:23:10 dominicm @seancorfield my implementation of this in user space might interest you. You set properties using jvm opts, then have a main which reads those properties and runs evaluations from them. #2020-05-16 23:41:00 alexmiller Early versions of clj did this :) #2020-05-16 22:23:33 dominicm Implementation is a little strong perhaps. Idea :). #2020-05-16 23:01:33 alexmiller Re init opts, at some point you’re programming, just write a clj and call it #2020-05-16 23:46:33 seancorfield Well, that was why, in the end, we wrote our little dev/repl.clj script at work -- it looks at what's on the classpath and starts a Socket REPL, an nREPL Server, REBL, or a regular REPL. #2020-05-17 00:19:20 alexmiller There are a couple generic ideas we have in this area that we will be working on, not sure yet if they are clj or clojure.main #2020-05-17 06:37:15 dominicm The composition is important I think. I don't want to maintain a clj for every project I work on. #2020-05-17 08:22:31 didibus I find writing a little clj file for custom stuff pretty convenient. I wouldn't say no to more systematic approaches if clj adds them. Just saying, it's not that bad to have a .clj file #2020-05-17 08:23:39 didibus It means everyone who knows Clojure can understand the build script, since it's just a clj file. As opposed to having to learn some new configuration system. #2020-05-17 08:24:12 didibus The only downside is paying the startup price on every script execution. #2020-05-17 09:12:02 dominicm Different contexts I think. A project can have a single clj file, but a user might have many they compose in different ways. #2020-05-17 09:12:48 dominicm For example, in one project I might want to run a clj file which launches postgres in the background if it's not already started, but I want that to work alongside the project's clj file.
clj -i pg.clj -i theproject.clj
#2020-05-17 13:05:42 alexmiller clj is for running a (single) Clojure program. It sounds like you’re trying to run multiple programs. At that point, use bash or babashka or write a generic clj tool that launches multiple programs. #2020-05-17 13:44:31 dominicm I want to have side effects on the world before a program runs. Usually within the same jvm. #2020-05-17 17:26:09 wilkerlucio hello, I'm having some version conflicts problem in my project, and on trying to debug I remember on lein when we ask for the tree it suggests some exclusions to avoid dependency conflicts, is there any such tool like this for use with deps.edn dependencies? #2020-05-17 17:31:25 alexmiller depot can help with some similar things #2020-05-17 18:42:06 dominicm To just clarify what I was saying above ^ I'm mostly interested in dev tooling. Things like scope-capture, nrepl, etc. So I want to effect the running JVM. #2020-05-20 06:15:16 dharrigan is it necessary to include clojure as a dep in the deps.edn? I've seen a few projects with it included. #2020-05-20 10:34:19 dominicm There was a news-and-articles by daniel janus recently about this #2020-05-20 11:22:38 dharrigan ooh. ta. #2020-05-20 12:43:28 alexmiller Yes, but it is included by default #2020-05-20 12:48:07 alexmiller So I guess no :) #2020-05-20 12:49:11 dominicm My personal thoughts are that you should mark your minimum required version. Because if you don't update your computer (for some reason) and then rock up on a project using Clojure 1.22 features, and you've got 1.15 you'll be scratching your head for a while. One less thing to manage externally. #2020-05-20 13:47:11 dharrigan It's more to do with docker containers. If I can remove the dependency (as it's already there as part of openjdk-11-tools-deps-slim-buster), then I can remove it from the deps.edn file, thus making the binary ever so slightly smaller. #2020-05-20 13:56:37 dominicm Hmm, I think you'll only get one copy either way... #2020-05-20 14:01:32 alexmiller you need to have it, and you will via the install deps.edn, it's just a question of version #2020-05-20 14:01:52 dominicm Maybe by removing it, you're just forcing to match the version in your docker. #2020-05-20 16:12:13 dharrigan Probably safer to leave it in. #2020-05-20 16:12:18 dharrigan After all, storage is cheap 🙂 #2020-05-20 07:41:13 Piers Hello, I've seen examples of installing dependencies using clj -Sdeps '{:deps {bidi {:mvn/version "2.1.6"}}}' but I was wondering if it's possible to instruct clj to install dependencies listed in deps.edn without doing anything else? Similar to lein deps I suppose. #2020-05-20 07:52:27 delaguardo clj -Spath as an option. it will download dependencies and print out calculated class-path #2020-05-20 11:38:36 seancorfield I think I've seen clj -e nil given as a quieter example. #2020-05-20 12:24:26 teodorlu I've used clj -e :deps-download-complete when building Docker images #2020-05-20 21:45:23 Piers Thanks for the suggestions, that's really helpful. My use case is also for building Docker images. #2020-05-20 17:42:47 dvingo is there a way via clj to see all available aliases? #2020-05-20 17:46:56 alexmiller Not currently. I’ve been thinking about adding thAt #2020-05-20 17:50:32 dvingo np, thanks #2020-05-20 18:46:37 avi I apologize for the probably FAQ, but I have searched for this… when I specify a gitlib as a dependency, is there a way to specify the path within the gitlib to the source code? e.g. there’s a dep I’d like to use which has its Clojure code in src/clj/ #2020-05-20 19:07:17 dominicm @aviflax 👋 That would be :deps/root :) #2020-05-20 19:10:06 avi 👋 Hi Dominic, thanks!
I saw :deps/root in the docs, but the description therein says:
> Specifies the relative path within the root to search for the manifest file.
In the case of this particular dep, the manifest file doesn’t even really exist… I’m faking it by manually setting :deps/manifest to :deps, to finagle tools.deps into treating this dep as if it has an empty deps.edn, which works in this case because the dep has no dependencies itself.
Anyway, that’s why I didn’t try it, but it makes sense that it would also change how the lib is added to the classpath. I’ll try it. Thanks! #2020-05-20 19:12:05 avi Hmm that’s not quite working. I set :deps/root to src/clj but the path added to the classpath is /src/clj/src 😞 #2020-05-20 19:15:18 dominicm Oh I see "to the source code". Yeah, that makes sense. I'm not sure you can get away with that. #2020-05-20 19:16:30 avi I figured. I kinda wish there was a way to override that src segment that tools.deps adds to the end of the root. Something like :deps/code-path or something. #2020-05-20 19:16:56 dominicm It would probably be :deps/paths or something :). #2020-05-20 19:18:28 avi Thanks for the link, but I’m not really understanding that function. I think you’re pointing out that there isn’t currently a way to do this? #2020-05-20 19:18:50 avi I was thinking maybe :deps/paths but I’m not sure about the plural part. I’m not sure it makes sense to add multiple entries to the classpath for a dep. #2020-05-20 19:24:09 hiredman :deps/root #2020-05-20 19:24:24 avi Tried that already. #2020-05-20 19:24:32 avi Doesn’t work in this case, see above. #2020-05-20 19:26:08 dominicm @aviflax I am saying there's no way to do this currently :) #2020-05-20 19:26:28 avi Yup, got it. Thanks though! #2020-05-20 19:26:56 avi I guess I’ll open a ticket/issue as a feature request, couldn’t hurt. #2020-05-20 19:28:47 alexmiller I think this already does what you want #2020-05-20 19:29:06 avi Oh! What am I missing? #2020-05-20 19:29:09 alexmiller or rather let me back up... #2020-05-20 19:29:19 alexmiller what kind of project is the git dep? #2020-05-20 19:29:30 alexmiller what kind of manifest file (pom, lein, deps) #2020-05-20 19:29:36 avi project.clj #2020-05-20 19:29:41 avi but it has no deps #2020-05-20 19:29:51 avi so I’m specifying :deps/manifest as :deps #2020-05-20 19:30:01 alexmiller that should just work - what doesn't work? #2020-05-20 19:30:35 avi In the dep’s dir structure, the Clojure code is in `src/clj/` #2020-05-20 19:30:41 alexmiller ah, ok #2020-05-20 19:31:02 avi I tried setting `:deps/root` to `src/clj` but the path added to the classpath was then `/src/clj/src` #2020-05-20 19:31:05 alexmiller yeah, there is no way right now to externally set a git dep's paths #2020-05-20 19:31:10 avi ah, ok #2020-05-20 19:31:17 avi would a ticket be helpful? #2020-05-20 19:31:21 alexmiller sure #2020-05-20 19:31:31 avi 👌 #2020-05-20 19:31:45 alexmiller you coudl fork it and add a deps.edn though #2020-05-20 19:32:05 avi ah, good point. Maybe I will. Thanks! #2020-05-20 19:32:21 alexmiller or clone and do the same with a local project #2020-05-20 19:32:52 avi I was thinking maybe I’ll open a PR to contribute the deps.edn to the main project; I’d rather not use a fork indefinitely #2020-05-20 19:35:50 alexmiller or that #2020-05-20 19:54:33 alexmiller I think the question presupposes the need to do this, but maybe the broader request is about supporting lein projects #2020-05-20 19:54:51 avi yeah #2020-05-20 19:54:54 avi agreed! #2020-05-20 19:55:05 alexmiller or even more broader, supporting extensible project types, one of which could be lein #2020-05-20 19:55:49 alexmiller tools.deps is actually written to support this but clj does not expose that capability currently (b/c there are a lot of questions about how to do it well) #2020-05-20 19:56:07 avi I can see it being a kind of support nightmare #2020-05-20 19:56:28 avi I’m sure a million edge cases would pop up #2020-05-20 19:57:49 avi maybe once tools.deps drops the “alpha” — then there could be a subsequent alpha/beta/whatever release that would enable the feature, with the caveat that it’s unstable, only for testing, etc? #2020-05-21 20:37:53 dominicm Silly question, what does override-deps do that extra-deps doesn't? I realized I've been using extra-deps in my deps.edn when I want to override a dep. #2020-05-21 20:46:43 alexmiller override replaces, extra adds #2020-05-21 20:46:57 alexmiller override does not add #2020-05-21 20:47:09 alexmiller that is, it is ignored if you don't already have the lib #2020-05-21 20:49:04 alexmiller I guess it would be more accurate to say override replaces and extra adds or replaces #2020-05-21 20:51:09 alexmiller further refining...
• extra adds new top-level deps (which may effectively replace existing top or transitive deps)
• override replaces the version of an existing top or transitive dep #2020-05-21 20:52:04 alexmiller in both cases, you should be guaranteed to get the version you've said - in extra, because it's at the top level, and in override because it replaces at all levels #2020-05-21 21:29:34 seancorfield @dominicm For a concrete use case, we use :override-deps heavily at work to "pin" versions of libraries across our entire repo. If a particular app doesn't use a given dependency, the :override-deps does nothing. For deps we expect to be overridden, we specify the coords as just {} (since they'll be supplied via the override). Does that help? #2020-05-21 21:35:49 dominicm Cool, thanks for both your input. 😁 #2020-05-26 12:49:56 Audrius Hi, I have
Error building classpath.
java.lang.NullPointerException
in one project when running what ever. I believe there could be a bug int the deps tool
Here is source: https://github.com/audriu/fake #2020-05-26 12:55:09 dpsutton Your figwheel main has “mvn/version” as a symbol rather than a keyword #2020-05-26 13:07:50 alexmiller I’ll take a look at why you’re seeing that error message, I would expect better #2020-05-26 13:08:49 Audrius I am very happy to help at least by contributing this 😎 #2020-05-26 13:40:23 dominicm Full stack:
java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
at java.util.regex.Matcher.reset(Matcher.java:309)
at java.util.regex.Matcher.<init>(Matcher.java:229)
at java.util.regex.Pattern.matcher(Pattern.java:1093)
at clojure.core$re_matcher.invokeStatic(core.clj:4856)
at clojure.core$re_find.invokeStatic(core.clj:4898)
at clojure.core$re_find.invoke(core.clj:4898)
at clojure.tools.deps.alpha.util.maven$version_range_QMARK_.invokeStatic(maven.clj:250)
at clojure.tools.deps.alpha.util.maven$version_range_QMARK_.invoke(maven.clj:248)
at clojure.tools.deps.alpha.extensions.maven$eval703$fn__706.invoke(maven.clj:46)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$canonicalize_deps$fn__1136.invoke(alpha.clj:69)
at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:49)
at clojure.core.protocols$fn__8140.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__8140.invoke(protocols.clj:75)
at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6828)
at clojure.core$reduce.invoke(core.clj:6810)
at clojure.tools.deps.alpha$canonicalize_deps.invokeStatic(alpha.clj:68)
at clojure.tools.deps.alpha$canonicalize_deps.invoke(alpha.clj:66)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:236)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:217)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:55)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:47)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:81)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:72)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:101)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:95)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:146)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:118)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
#2020-05-26 13:41:05 alexmiller Thx #2020-05-26 13:41:34 dominicm Oh, I see the problem #2020-05-26 13:42:03 dominicm I'm guessing the ns code doesn't care about keyword vs symbol though #2020-05-26 13:43:34 dominicm Yeah, it gets through coord-type because of the way it's written. But then version can't be read at any point, because it's not a keyword. #2020-05-26 13:44:16 dominicm There's also the fact it's figheel-main instead of fig*w*heel-main :) #2020-05-27 17:05:17 alexmiller added some error checking for this for next release, now you'll see:
Error building classpath. No :mvn/version specified for com.bhauman/figheel-main
#2020-05-26 13:44:54 dominicm @stebokas also, you've typo'd figwheel-main :)
- com.bhauman/figheel-main {mvn/version "0.1.9"}
+ com.bhauman/figwheel-main {:mvn/version "0.1.9"}
#2020-05-26 13:46:28 bhauman the most common typo ever 🙄 #2020-05-27 13:17:05 respatialized Is there a way using deps.edn to add code that gets eval'd on startup for an alias or for a repl launched using clj? The problem I'm specifically trying to solve is requiring jansi-clj for any terminal clojure session so it picks up my system terminal colors, but I was wondering if there was a generally recommended approach to the problem of "default startup code" using tools-deps. #2020-05-27 13:18:57 alexmiller you can use clojure.main's -i or -e sometimes for things likee this #2020-05-27 13:19:09 alexmiller clj -h will tell you more #2020-05-27 13:19:27 alexmiller -i loads an arbitrary clj file, -e runs an expression #2020-05-27 13:22:02 respatialized got it, so it's the simple matter of combining -e (require 'jansi-clj.auto) -r to launch a repl with some short startup code, and -i "/path/to/file" -r for anything longer. thanks! #2020-05-27 13:23:02 alexmiller yep, and you can put that in an alias with :main-opts #2020-05-27 13:23:22 alexmiller so clj -A:repl or whatever #2020-05-29 15:17:18 jjttjj I'm trying to use andare, a replacement for core async with the same namespace hierarchy, in a project that also depends on things which depend on core.async (shadow-cljs specifically). However just adding core.async to the :exclusions for shadow-cljs seems to prevent it's namespaces which use core.async from working. Is there something I'm missing? #2020-05-29 15:21:02 seancorfield @jjttjj Maybe shadow-cljs is AOT'd? That would be my guess. (your problem doesn't sound like it would be specific to Clojure CLI / deps.edn / tools.deps.alpha?) #2020-05-29 15:24:45 thheller it is only AOT if you request the AOT variant #2020-05-29 15:24:57 thheller I'm guessing that andare just isn't fully compatible? how does it fail? #2020-05-29 15:28:33 jjttjj File: jar:file:C:\Users\justin\.m2\repository\thheller\shadow-client\1.3.2\shadow-client-1.3.2.jar!/shadow/util.cljs
failed to require macro-ns "shadow.util", it was required by "shadow.util"
Error in phase :compile-syntax-check
RuntimeException: No such var: ioc/state-machine
#2020-05-29 15:31:10 thheller guess that doesn't exist in andare? it is an internal cljs.core var #2020-05-29 15:31:56 jjttjj Yup, I see it doesn't now, thanks #2020-05-29 15:32:33 jjttjj It looks like shadow.util in shadow-client uses some core.async internals #2020-05-29 15:35:34 thheller it does. I can probably remove it but I'm not sure trying to replace the entire core.async impl is sustainable #2020-05-29 15:35:53 thheller you are doing this for self-hosted CLJS right? #2020-05-29 15:37:04 jjttjj yes. I wouldn't necessarily request that either, just messing around right now 🙂 #2020-05-29 15:39:24 jjttjj although it looks possible that your go! macro that requires it might not be used much at all in shadow #2020-05-29 15:43:51 ghadi oh look, an incompatible thing with the same name #2020-05-29 15:45:23 seancorfield ^ That was my first thought too. Any "replacement" really should use at least a different root namespace, precisely to avoid these sorts of conflicts 😐 #2020-05-29 15:47:14 dpsutton i think its so self-host can use it. it needs to be named the same #2020-05-29 15:47:16 jjttjj makes sense. I'll prob just fork andare and rename things #2020-05-29 16:09:19 dpsutton rename to what? #2020-05-29 16:09:49 dpsutton > Andare preserves the namespaces present in core.async. Thus, bootstrap-compatible ClojureScript code that makes use of core.async can operate in self-hosted environments if you make the Andare artifact available for loading in lieu of the official core.async artifact. #2020-05-29 16:11:00 seancorfield Also, none of this seems relevant to this channel's topic -- am I missing something there @jjttjj? #2020-05-29 16:13:34 jjttjj Nope, sorry about that. Thought it might be tools.deps specific at first (relating to exclusions) but realize now that's not the case #2020-05-29 21:39:45 seancorfield Tag it with the tools.deps tag... #2020-05-29 21:40:56 lvh ah, I thought that was just for discussion, not for bug reports #2020-05-31 09:39:29 p-himik Let me know if this is not the right place for this question.
I'm trying to build an uberjar with depstar. It's my (almost) first attempt to do something like that.
Going by this snippet from the README:
# generate pom.xml (or create it manually)
clojure -Spom
# build the uberjar with AOT compilation
clojure -A:depstar -m hf.depstar.uberjar MyProject.jar -C -m project.core
# Main-Class: project.core
java -jar MyProject.jar
Did all that in a very simple "hello world" project.
Running that gives me
Error: Could not find or load main class project.core
Caused by: java.lang.ClassNotFoundException: project.core
I opened the jar with jd-gui and I see that there's indeed no project.core class even though the manifest mentions it.
Running a non-AOT variant with java -jar MyProject.jar -m project.core works just fine.
Are there some obvious steps that I might have missed? Or can it be an issue with depstar? #2020-05-31 09:45:17 seancorfield Is your project on GitHub so I can take a look @p-himik? #2020-05-31 09:46:28 seancorfield Does your project.core namespace have (:gen-class) in the ns form? Does it have a -main function? #2020-05-31 09:47:13 seancorfield (if you create an app project with clj-new, it'll have all the right bits in it that you need for an uberjar -- it'll even have the :aliases already set up in deps.edn) #2020-05-31 10:22:56 p-himik (:gen-class) seems to be it, thanks! I had no idea it was needed for -main with AOT. I'll try clj-new as well. #2020-06-01 05:18:54 didibus Would it make sense to add a npm resolver? #2020-06-01 05:20:43 didibus Right now, it seems a ClojureScript lib can't depend on another ClojureScript lib that depends on a JS lib unless someone repackages the JS lib into Maven. But since tools.deps has custom resolvers, if it could resolve from npm, it seems it would allow this scenario to work #2020-06-01 05:23:59 seancorfield When you say "Maven", does that include the cljsjs infrastructure or whatever it is? #2020-06-01 05:25:50 didibus Ya it does #2020-06-01 05:26:54 didibus cljsjs just repackages NPM packages into Clojars #2020-06-01 05:27:35 didibus As it does so, it might also go ahead and perform some minification on it, and it'll add the required extern files. #2020-06-01 05:29:16 didibus So cljsjs is convenient when you're too lazy to minify or write extern files yourself. But right now, it is also mandatory just to be able to resolve deps and deps conflicts. At least this is my understanding. #2020-06-01 05:32:03 didibus I guess the challenge of a NPM resolver would be transitive NPM deps. Tools.deps would need to understand the npm package.json (the equivalent of pom.xml) and all that #2020-06-01 05:36:03 seancorfield I would imagine the #clojurescript channel would be the best place to petition for such a thing? How does David Nolen feel about cljs dependencies from the NPM/JS world? I saw the recent changes to make it easier to publish cljs stuff to NPM (as I understood the news)... #2020-06-01 05:49:56 didibus Hum, ya not sure. From my understanding, the new bundle target isn't really about publishing to NPM.
Basically, when you want to deploy a ClojureScript app, you need to do a sort of uberjar, but in JS world it's called a bundle.
The ClojureScript compiler has the capability to bundle your ClojureScript code and all its dependencies together. Since all users will need to download your bundle when viewing your site, bundle size is a sensitive topic.
ClojureScript code and ClojureScript dependencies as well as JS dependencies that are designed with Google Closure support can be bundled by the Google Closure compiler in a super optimized way (in terms of having a very small size). It renamed all vars to super short names, removed all whitespace, and does dead code elimination at the function and variable level.
That said, JS dependencies not designed for Google Closure can't go through it. So ClojureScript compiler lets you specify those as "foreign-libs". In which case, it will still bundle them up, but it won't optimize them at all.
For a project that depends on a lot of such library, that isn't very good, because it still results in a large bundle size.
There exist other bundlers for JS, which can take those type of libraries and bundle them with a reduced size. But until now, if you tried to used them to bundle the already bundled output of the ClojureScript compiler, it would fail.
The new bundle target for the ClojureScript compiler fixes this. It makes sure to output a bundle that can be further bundled by JS bundlers.
Nothing about it really has anything to do with package management. At least this is my current understanding. #2020-06-01 17:17:31 seancorfield @U0K064KQV Thanks for the explanation. Some of that I already knew but I was hazy on the relationship between cljs and bundles. One day I hope to find the time/bandwidth to dig back into cljs and maybe build something interesting so I can learn about the tooling available now (2014 was the last time I tried to build anything serious in cljs and a lot has changed since then! 🙂 ) #2020-06-01 21:45:02 didibus My pleasure, sometimes I write detailed explanations but it's really for myself that I do it. Forces me to put some order in my understanding and tests also whether I'm still missing some bits. #2020-06-01 07:55:07 thheller FWIW I don't think it make sense to move this into deps.edn. I doubt Alex wants to maintain and interact with the insanity that is npm. We already have the mechanism to declare npm dependencies and installing them in CLJS directly, just a bit basic maybe. can always add more options to that when needed. #2020-06-01 08:23:54 didibus Is that through shadow-cljs? Or without? #2020-06-01 08:28:49 thheller CLJS has --install-deps via cljs.main and shadow-cljs just does it automatically on startup #2020-06-01 08:35:29 didibus Hum, so there's something I don't understand then. Install-deps is unaware of versions. And it does no transitive deps handling. Or I haven't figured out how #2020-06-01 08:36:37 didibus So for example, if lib A wants to declare some npm-deps, and app X wants to use lib A, app X won't automatically pull down lib A's deps #2020-06-01 08:39:20 thheller in CLJS automatically no, you have to run --install-deps in app X #2020-06-01 08:39:27 thheller in shadow-cljs it is automatic #2020-06-01 08:40:18 didibus Well, I'll definitly be trying shadow-cljs eventually. But for now, --install-deps, even run manually doesn't pull transitive dependencies for me #2020-06-01 08:40:21 thheller both look for deps.cljs files in the libraries and use :npm-deps {"some-lib" "some-version"} to install #2020-06-01 08:40:40 didibus Oh, what is a deps.cljs file 😛 #2020-06-01 08:40:49 thheller well do this libs you are testing contain the proper deps.cljs? many that still just depend on cljsjs don't #2020-06-01 08:41:11 didibus I am writing the lib. First time I hear about a deps.cljs file #2020-06-01 08:45:12 thheller hmm that doesn't even mention npm-deps #2020-06-01 08:45:31 thheller maybe there are no docs for this? I always thought there were #2020-06-01 08:46:10 didibus There doesn't seem to be. And I felt like I had gone over most of the docs looking for something like this. Maybe it just needs some better doc #2020-06-01 08:47:09 didibus So in shadow-cljs you just leverage the compiler install-deps feature under the hood? Or you make use of npm directly? Or you implemented a commonJS repository resolver of your own? #2020-06-01 08:48:09 thheller no I don't use the CLJS install-deps ... mostly because it didn't exist when I wrote the feature for shadow-cljs #2020-06-01 08:48:34 thheller I don't handle any package installing whatsoever, it just runs npm install nothing more. #2020-06-01 08:48:59 thheller (same as CLJS does nowadays for --install-deps) #2020-06-01 08:49:27 didibus Ok, and you grab the-package and the-version from the deps.cljs file on the resource path? #2020-06-01 08:54:33 didibus Just trying to figure out how to best package my lib. So if I make use of a npm deps, I just declare it in the deps.cljs along with foreign-libs and externs. And now anyone would be able to just add a dependency to my lib and unless they have a newer version of the npm deps themselves, everything would work as is and they'd get my lib and my transitive npm deps (assuming shadow or needing to run install-deps manually otherwise) #2020-06-01 08:56:58 thheller generally it is either/or. either you depends on :npm-deps OR you use foreign-libs. don't mix them. #2020-06-01 08:57:07 didibus And where am I supposed to put the deps.cljs ? That weird doc mentions inside a src folder? But I assume just anywhere on a resource path of my deps.edn? Or should it be project root? #2020-06-01 08:57:57 thheller the file needs to be in the published .jar file at the root #2020-06-01 08:58:17 thheller so generally any source-path will do #2020-06-01 08:58:42 didibus Ok, and if I'm temporarily using a local dep? Same logic applies? #2020-06-01 08:59:51 thheller for CLJS don't know. for shadow-cljs yes, it just loads any deps.cljs files on the classpath #2020-06-01 09:00:22 didibus :+1: alright, will try it out. Thanks. #2020-06-01 18:09:24 alexmiller the idea of using npm to find transitive cljs deps has come up here several times and it's interesting but needs better thinking starting from the goals. tools.deps really has several parts and it seems like the relevant part here is using the tools.deps dependency expander to trace the transitive set of dependencies, starting from a cljs project. tools.deps is designed to be extended for this by implementing various multimethods in the ctda.extensions ns. I've looked at the npm repo and it has everything you would need to do so and would be pretty simple to plug in in that way. #2020-06-01 18:10:22 alexmiller however, it is then at odds with the rest of tools.deps (and clj) in that the other parts are about making a jvm classpath and I don't think that's what you need or want for cljs. #2020-06-01 18:12:21 alexmiller so yes, you could use tools.deps to get a transitive dependency expansion from a heterogenous set of sources including maven, local, git, and npm. I think you want to do that from some cljs tool that then knew what to do with all that stuff to appropriately run the cljs compiler (or whatever). #2020-06-01 18:14:08 alexmiller if that big picture made sense, I'd be happy to lay out what needs to be done to plug in an npm procurer into the tda extensions. That part is not hard, but it's not useful unless this in service of some greater vision. #2020-06-01 19:04:38 dominicm I think npm is useful outside of cljs. It would make a good alternative to webjars. #2020-06-01 19:24:27 alexmiller useful to do what #2020-06-01 19:50:52 dominicm I use webjars to serve assets (js, css) for libraries like codemirror #2020-06-01 21:59:08 didibus Woa, webjars already mirrors all of npm and Bower into Jars? #2020-06-01 21:59:31 didibus That means I'm theory, we can already use tools.deps.to.depend on those. #2020-06-01 22:00:33 didibus The only missing piece is from the cljs compiler. But you could easily write a tools.deps alias that extracted the content from the jar and put them in a node_modulrs folder and then created the deps.cljs file. So you could run that, and then run the compilation #2020-06-01 22:00:50 didibus Have you found webjars is up-to-date with npm? #2020-06-01 22:00:58 didibus And mirrors absolutely everything? #2020-06-01 22:18:43 dominicm It's an automated process to request more #2020-06-01 20:05:01 hiredman those are different things though, one is providing dependencies to the cljs compiler, the other is providing runtime resources #2020-06-01 20:11:38 hiredman it seems like tools.deps is mostly about runtime resources (building a classpath), it just happens that those things almost always overflap significantly in clojure, and people have built tools on top of tools.deps that reinterpret it as build time dependencies (building uberjars out of the stuff it puts on the classpath) because of that overlap #2020-06-01 20:12:41 hiredman there is, from my very limited perspective on cljs, a lot less overlap between compile time and runtime dependencies there because of the split between the compiler and runtime, and the way js is usually delivered #2020-06-01 20:26:10 hiredman thinking about it, it also strikes me as very unlikely that packages right from npm are suitable to be served up ala webjars, my guess is you need some kind of intervening build step, so really both cases are inputs to some build/packaging process #2020-06-01 21:51:03 didibus To me, this is the same. We want to provide the ClojureScript compiler (which is a Clojure application) the necessary resources that it needs. Because it relied on npm currently, it makes an assumption to not look for them on the classpath, but instead look for them in a node_modules folder. But I feel if it could leverage tools.deps, it's not unrealistic for it to look for them on the classpath instead. With target :bundle it could create a node_modules folder of symlinks to the deps pulled down by tools.deps and thus allow JS bundlers to "find" the tools.deps managed JS dependencies.
Or maybe, tools.deps could just pull the npm deps into that folder directly. Not sure if it's a core part of tools.deps to put all the deps in some central place? Or if it would be fine for it to put them in different places? #2020-06-01 22:26:57 alexmiller It‘a not - it’s up to the procurer to do that, and then tell tools.deps where it is #2020-06-02 00:22:51 didibus That makes it even better then. Anyways, I'm just plenting the seed of an idea. See where things go. #2020-06-01 21:54:21 didibus Also, I'm not super familiar with JS bundlers, do they expect to bundle node_modules stuff? Or could the compiler just copy over all the necessary JS on the output folder and the JS bundler can just bundle it up. #2020-06-01 21:54:22 hiredman they are not dependencies of the clojurescript compiler, they are dependencies of the code it is compiling #2020-06-01 21:54:42 didibus Right, I see them as resources needed by the ClojureScript compiler. I was using dependencies loosely here. #2020-06-01 21:59:37 hiredman sticking things on the class path because we have a tool for building class paths does not seem like a considered approach #2020-06-01 22:04:31 hiredman it sounds like the problem you want to solve is transitive js dependencies #2020-06-01 22:05:03 didibus Well, personally, I find an explicit list of paths to resources is more considered then an implicit one. But NPM already made that choice. Though here it seems we could make a different one. The only difference is where to look for the resources, do you just scan the classpath, or do you also look for a specific relative folder.
That said, it seems @alexmiller was saying that it would be possible that tools.deps doesn't do classpath generation for those, but just feeds the resolution to something else. So that could be an option as well. So maybe that something else can generate the node_modules folder instead.
That said, I think going with the classpath is best. Because cljs already uses the classpath for ClojureScript deps, and for cljsjs deps. So reconciling those two seem like a nice outcome. #2020-06-01 22:10:54 didibus I'm trying to solve JS dependency resolution. So pulling down all required direct and transitive JS dependencies, detecting conflicts between them and letting the user resolve those. That's the first part. Feel tools.deps could do this. Right now the ClojureScript compiler itself is slowly building this functionality inside itself (though it delegates to npm mostly).
The second issue is setting up the environment for running the ClojureScript compiler where it can find your dependencies. It currently relies on a mixture of the classpath + looking for an implicit node_modules folder. I'm not as adamant here, but I still feel if we moved the nodes deps to the classpath as well, it seems to simplify things. Also, it gives us a single place for ordering in terms of which lib to grab first if the JS class exists in more than one place. #2020-06-01 22:11:48 didibus But probably we should discuss these two things separately. #2020-06-03 18:48:44 vlaaad not sure it's important, but I was reminded of a small caching problem I had with tools-deps: it seems to cache the classpath per list of aliases enabled or something similar that excludes the java version, while maven allows selecting different dependencies based on java version right inside the pom #2020-06-03 18:49:52 vlaaad so the same deps.edn might produce different classpaths based on JDK used #2020-06-03 18:51:29 ghadi a repro would be helpful. By design tools.deps doesn't attempt to mimic Maven #2020-06-03 18:52:10 ghadi that is not a repro, that's a repo #2020-06-03 18:52:24 vlaaad good point #2020-06-03 18:52:34 ghadi "`clojure -A:foo -Spath` -> expected this, but received that" is a repro #2020-06-03 18:52:46 vlaaad ah, sorry, I'll make a repro #2020-06-03 19:02:32 vlaaad repro #2020-06-03 19:07:45 alexmiller "switch java version to 8 so it should trigger different profile according to activation rules" is not something clj does #2020-06-03 19:07:51 alexmiller that's a maven thing #2020-06-03 19:09:25 alexmiller you can make Clojure aliases that include the desired classifier variant of cljfx if you like #2020-06-03 19:11:04 vlaaad but it's what it does? #2020-06-03 19:11:13 alexmiller it's what Maven does #2020-06-03 19:11:38 alexmiller clj is not Maven and I have no plans to add that #2020-06-03 19:12:58 vlaaad sure, I understand that is abusing maven features #2020-06-03 19:13:18 alexmiller it's very much using Maven features as they were intended #2020-06-03 19:13:54 alexmiller just don't expect clj to support all of that (this is just the tip of the iceberg) #2020-06-03 19:17:44 vlaaad Okay, thanks #2020-06-03 22:08:08 drewverlee I have deps that specifies a private github repo via SSH (e.g /cdn-cgi/l/email-protection) This works finally locally. But i'm trying to know run my build scripts as part of a deployment pipline (specifically google steps). I'm running into some complications concerning deployment keys, ssh, etc... by any chance has anyone been through this before and have any advice? #2020-06-03 23:44:36 hiredman It may be easier to use the git build step to fetcch the repo and then treat it as a local dep #2020-06-04 00:52:32 drewverlee The downside is that you have to have a GitHub deploy key per repo. #2020-06-04 00:09:39 kenny Git deps won't work at all unless they generate an older key. I think anyone with a new Mac will hit this issue 😞 #2020-06-04 00:39:22 ghadi ed25519 keys work fine if you use an ssh-agent #2020-06-04 00:39:37 ghadi If you rely on .ssh/config you will have a bad time #2020-06-04 14:09:18 drewverlee can you point to something that explains why using a config will lead to a bad time? #2020-06-04 14:09:36 drewverlee This seems relevent to some of the waters i'm having to navigate. #2020-06-04 14:09:51 drewverlee As i removed the config and things "just worked" #2020-06-04 14:49:04 drewverlee How can i get more information about why the connector is available?
Cloning:
In the same context i can run git clone on a repo and it succeeds. #2020-06-04 14:53:51 drewverlee maybe the docker image i'm running in doesn't have something git deps expects to be installed at a system level? #2020-06-04 14:54:26 alexmiller are you running ssh-agent? #2020-06-04 14:55:40 drewverlee checking... #2020-06-04 14:59:10 ghadi echo $SSH_AUTH_SOCK will tell you definitively #2020-06-04 14:59:37 ghadi if you're running in a docker container, probably not, unless you started it explicitly #2020-06-04 15:01:45 drewverlee thanks ghadi. #2020-06-04 15:15:45 drewverlee it doesn't appear to output anything. argh, ill have to somehow start it and save it between google cloud steps. #2020-06-04 15:41:52 drewverlee yea. i just jump started the ssh agent first and then called clojure #2020-06-10 00:32:40 porkostomus I can use the CLI to launch the Liquid Clojure editor from Windows Powershell, but only if I escape the double-quotes:
PS C:\Users\admin> clj -Sdeps '{:deps {mogenslund/liquid {:mvn/version \"RELEASE\"}}}' -m liq.core
It fails however if I try to use the alias:
PS C:\Users\admin> clj -A:liquid
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate dk/salza/liq/core__init.class, dk/salza/liq/core.clj or dk/salza/liq/core.cljc on classpath.
Full report at:
C:\Users\admin\AppData\Local\Temp\clojure-4978726344790284382.edn
My deps.edn entry:
:liquid
{:extra-deps {mogenslund/liquid {:mvn/version "RELEASE"}}
:main-opts ["-m" "dk.salza.liq.core"]}
All the other aliases seem to work fine. #2020-06-10 00:41:45 seancorfield @porkostomus Looks like the project changed it's main namespace (from dk.salza.liq.core to just liq.core) #2020-06-10 00:42:30 seancorfield I'll update my dot clojure file #2020-06-10 00:45:00 seancorfield @porkostomus I fixed the alias in my repo -- if you cloned that, you can just git pull to get the latest #2020-06-10 00:56:57 porkostomus @seancorfield Great, thanks! #2020-06-10 21:34:53 Aaron Cummings I have just upgraded from 1.10.0.442 to 1.10.1.536, and I've found a change in behavior. In .442 I had changed my $INSTALL/lib/clojure/deps.edn to point the central and clojars repos to a local partial mirror. In .536, the same change has no apparent effect. Here is an example of my lib/clojure/deps.edn:
{
:paths ["src"]
:deps {
org.clojure/clojure {:mvn/version "1.10.1"}
}
:aliases {
:deps {:extra-deps {org.clojure/tools.deps.alpha {:mvn/version "0.8.677"}}}
:test {:extra-paths ["test"]}
}
:mvn/repos {
;; "central" {:url ""}
;; "clojars" {:url ""}
"central" {:url "file:///home/aaron/cljtest/central/"}
"clojars" {:url "file:///home/aaron/cljtest/clojars/"}
}
}
The reason I'm doing this is our development machines at work do not have internet access, so I determine in advance all of our central and clojars dependencies and cache those locally in NFS.
Is there a different way that I should override central and clojars default locations? #2020-06-10 21:37:06 alexmiller that file is no longer used - it's embedded in a resource in the uber jar #2020-06-10 21:37:20 alexmiller you can set those in your ~/.clojure/deps.edn though and should have the same effect #2020-06-10 21:42:24 Aaron Cummings Okay, I'll give that a try. #2020-06-10 21:43:56 alexmiller (that would have worked on older version too) #2020-06-10 21:44:49 Derek Passen on that note, when I run clj -Sverbose on 1.10.1.536, the config_paths still lists the system install deps.edn file #2020-06-10 21:44:49 Derek Passen on that note, when I run clj -Sverbose on 1.10.1.536, the config_paths still lists the system install deps.edn file #2020-06-10 22:01:01 alexmiller yeah, it's hard-coded in there and hasn't been updated #2020-06-10 22:02:11 alexmiller there are actually some tools that rely on this output and do (now incorrectly) still use the file so I have been delaying on removing it #2020-06-10 22:02:26 Derek Passen makes sense #2020-06-10 21:51:51 Aaron Cummings Super, that did work. You're right @dpassen1 - the -Sverbose flag output did lead me astray.
I did like the file in the installation - that let me setup for all of my users in one place. But I'll switch us over to the ~/.clojure/deps.edn instead. #2020-06-10 21:57:32 seancorfield Given that the system deps is meant to be stable and immutable, I think it's good for it to no longer be an actual file. #2020-06-10 21:58:07 Derek Passen Agreed Sean #2020-06-10 21:58:41 Derek Passen There’s a reason for ~/.clojure/deps.edn and CLJ_CONFIG env var #2020-06-10 21:58:46 Derek Passen at least i think it’s CLJ_CONFIG #2020-06-10 21:59:45 seancorfield It is. We use that at work to point to a repo-wide deps file, and then each subproject in the repo has its own deps file. #2020-06-10 22:00:34 Derek Passen I probably learned it from your mention somewhere 🙂 #2020-06-10 22:00:41 seancorfield CLJ_CONFIG=../versions clojure -A:defaults ... -- the :defaults alias pulls in all the :override-deps from versions/deps.edn #2020-06-10 23:31:51 Aaron Cummings I had forgotten about CLJ_CONFIG - that seems like the better solution to what I'm doing. Thanks for the pointers everybody. #2020-06-11 19:27:02 nha I am looking for something like a maven api that I can call from a Clojure app to make a .jar from a java file + it’s main class name.
I’ve never used maven directly (at least not in anger). Any opinions here?
For context: I am making a .jar and uploading it to a lambda-like service.
Not about tool deps so I hope this is OK - seems like the place where I am most likely to get answers 🙂
Please reply in the thread #2020-06-11 19:27:02 nha I am looking for something like a maven api that I can call from a Clojure app to make a .jar from a java file + it’s main class name.
I’ve never used maven directly (at least not in anger). Any opinions here?
For context: I am making a .jar and uploading it to a lambda-like service.
Not about tool deps so I hope this is OK - seems like the place where I am most likely to get answers 🙂
Please reply in the thread #2020-06-11 19:27:18 nha Please reply here to keep the channel tidy 🙂 #2020-06-11 19:28:33 dominicm @U0ALP2929 What service are you using? I created an AWS lambda-specific output for pack and would be interested in targetting others. #2020-06-11 19:30:03 nha So hence lambda-like, it’s not exactly the same. However they accept a
--jar target/my-jar-with-dependencies.jar \
--classname org.example.functions.WordCountFunction \
#2020-06-11 19:31:04 dominicm OK, looks like you need an uberjar of some kind in that case. I would recommend the library @U04V70XH6 maintains for that. #2020-06-11 19:31:47 dominicm There's downsides to uberjars (which is why pack doesn't produce them), I'm curious to know if they accept anything else, so I'm reading the docs now :) #2020-06-11 19:37:00 dominicm That's the one :) #2020-06-11 19:37:21 dominicm Doesn't look positive for alternative forms. Unfortunate. Depstar is what I'd recommend :) #2020-06-11 19:41:56 nha depstar looks like it is supposed to be used from the CLI, and not sure it supports .java files?
I am looking for something to embed. I could always shell it / reach into the namespaces though #2020-06-11 19:43:34 dominicm Oh, sorry, I thought you wanted to make a jar for clojure #2020-06-11 19:43:37 dominicm apologies #2020-06-11 19:43:43 dominicm Right, you don't need much at all #2020-06-11 19:43:58 dominicm javac MyFile.java then zip jar.jar MyFile.java #2020-06-11 19:44:01 dominicm that's it, you're done! #2020-06-11 19:44:11 dominicm there might be an official java tool for making a jar, but zip works OK #2020-06-11 19:44:13 seancorfield @U0ALP2929 To build uberjars with depstar, you should be able to depend on the library and call hf.depstar.uberjar/run with the appropriate options -- see how the -main function sets those up. It assumes there's a pom.xml file available because it uses the presence of that to trigger some of the AOT and manifest generation stuff. It assumes, at a minimum, you've run clojure -Spom in a project, but you could probably generate a stub pom.xml as a temp file and pass that in. #2020-06-11 19:45:15 seancorfield And depstar works from the classpath -- so it will build a JAR from whatever is on your classpath which is very likely not what you're looking for 🙂 #2020-06-11 19:45:55 seancorfield jar is the "official Java tool for making a jar" but it's basically just zip 🙂 #2020-06-11 19:46:10 nha Ahah I see 🙂 not what I am looking for indeed. Shelling to javac or even mvn looks like it could be the way then #2020-06-12 16:57:11 Travis I am currently attempting to use the new Google Artifact Maven Registry in Google cloud which has a custom maven wagon. Is this possible to use with tools-deps ? #2020-06-12 17:02:23 alexmiller if it needs a custom wagon, then no #2020-06-12 17:02:48 alexmiller why does it need a wagon? #2020-06-12 17:02:52 alexmiller got a link? #2020-06-12 17:11:20 Travis I believe its to handle the security mostly to connect to it. Uses the underlying gcloud ( cli ) creds #2020-06-12 17:13:12 alexmiller yeah, that was my guess, might be be some storage access optimizations over just http too #2020-06-12 17:13:31 alexmiller well, feel free to log this as a request at https://ask.clojure.org so people can vote on it #2020-06-12 17:13:47 Travis ok, thanks #2020-06-12 17:13:56 alexmiller if it's a wide enough need, will consider #2020-06-13 05:54:32 sveri Hi, Is there a way to make sure I download needed dependencies for a givem library? Like: clojure -Sdeps '{:deps {seancorfield/depstar {:mvn/version "1.0.94"}}}' but without opening a repl? #2020-06-13 06:06:28 seancorfield @sveri Add -e nil to the end of that command. #2020-06-13 06:07:02 seancorfield It will perform all the downloads etc and then evaluate nil (and not print it because it only prints non-`nil` values) #2020-06-13 06:08:05 seancorfield -Stree and -Spath will also circumvent a REPL and will show you the dependency tree or class path respectively, if you want some confirmation of the resources being accessed. #2020-06-13 06:09:38 sveri Thank you, exactly what I am looking for. I don't need the extra output for now. #2020-06-13 06:09:47 sveri @seancorfield #2020-06-13 08:20:30 sveri @seancorfield I hope it's fine if I ask you directly here. I have been using depstar successfully so far and am experimenting with graalvm right now. It seems like native-image analyzes faster if direct linking is enabled. Is there a way to pass the #2020-06-13 08:30:08 sveri Ah, I think I got it. deps.edn supports :jvm-opts ["-Dclojure.compiler.direct-linking=true"] for aliases, if I read that right. #2020-06-13 18:30:08 seancorfield @sveri Or you can pass it on the command-line as -J-Dclojure.compiler.direct-linking=true #2020-06-13 19:05:39 sveri @seancorfield Does that -J-D... work for clj calls too? In only saw it in reference to the native-image tool #2020-06-13 19:06:53 seancorfield Yes, -J is a clojure/`clj` option to pass JVM options. #2020-06-13 20:22:19 sveri Okay, great, thank you. #2020-06-15 22:44:04 alidlorenzo tools.deps doesn’t load :local/repo aliases, right? how do you handle conditionally loading dev/test/etc aliases on per subproject basis? #2020-06-15 22:44:42 alexmiller Right, and you don’t #2020-06-15 22:45:02 alexmiller Something we’ve thought about a little #2020-06-15 22:45:13 alexmiller But needs more thinking #2020-06-15 22:46:26 alidlorenzo yeah, I found some archived slack convos about it. kind of hard to appropriately work with monorepos without such a feature, everything has to configured in the root deps.edn (?) #2020-06-15 23:03:18 alidlorenzo what if it aliases were specifically included, just like dependencies are?
{my-subproject {:local/root "./my-subproject"
:aliases [:dev :test]}} #2020-06-15 23:19:49 alexmiller I’m not going to start from what if - this needs a design process #2020-06-15 23:25:45 seancorfield We have a monorepo at work and we specifically use CLJ_CONFIG to set the "user" deps.edn to come from a specific directory in the monorepo so that we have a single, central place for all repo-wide aliases anyone needs. This also helps maintain repeatability because each developer's "user" deps.edn is excluded and we can control exactly what each alias does. #2020-06-15 23:31:47 alidlorenzo if subprojects have paths that are loaded based on alias, do you configure this all in the central deps.edn instead?
e.g {:extra-paths ["subproject1/test" "subproject2/test"]} #2020-06-15 23:52:32 seancorfield It depends on whether the alias (and the optional subproject) are going to be needed for "all" other subprojects or just specific one(s). We scope things appropriately. #2020-06-15 23:53:02 seancorfield All our subprojects are at the same level so ../subproject/path always works. #2020-06-16 09:18:43 robert-stuttaford apologies if an FAQ (i did try to google this) - should :git/url work if the target sha is in a feature branch? so tools.gitlibs appears to be failing silently - situation: i see the gitlibs' "Checking out: " message, but no errors, and then when tools.deps tries to use the checkout, it fails because it's not there. i checked ~/.gitlibs manually and found the folder to be missing. #2020-06-16 09:58:40 borkdude @robert-stuttaford yes. have you tried -Sforce? I find myself needing that occasionally with git deps #2020-06-16 11:49:05 robert-stuttaford -Sforce doesn't seem to make a difference, unfortunately, thanks @borkdude #2020-06-16 11:51:29 alexmiller Might be something wrong with object dir - you could try rm’ing that under gitlibs #2020-06-16 11:53:47 alexmiller Can’t say I’ve seen this kind of problem before #2020-06-16 11:55:52 vlaaad maybe feature branch isn’t pushed? 🙂 #2020-06-16 11:56:59 alexmiller should fail if sha isn’t known #2020-06-16 12:19:30 robert-stuttaford i have one other hash in the folder, so it's defo succeeded at least once before. i'll scrub and retest! (feature branch defo pushed haha) #2020-06-16 12:22:07 robert-stuttaford the commit hash that succeeded is on the default github branch (`develop` in our case), whereas the one that failed is in a feature branch. could it be that the git checkout it's doing isn't able to find the feature branch's commit? #2020-06-16 12:24:00 alexmiller Shouldn’t matter at all, I do this all the time #2020-06-16 12:24:03 robert-stuttaford scrubbing and retesting yields the same results #2020-06-16 12:25:35 alexmiller You sure you have a commit sha and not like a tree sha or something #2020-06-16 12:26:50 robert-stuttaford Checking out:
the first line is one that works (it's there after i run), the second line is the one that's meant to work but doesn't (not there after i run), and the third line is the downstream failure due to the missing folder. #2020-06-16 12:27:26 robert-stuttaford i guess the next step is to dig with tools.gitlibs at the repl! #2020-06-16 12:27:42 alexmiller Yeah that definitely drops down a layer #2020-06-16 12:28:00 robert-stuttaford glad to know that this should work, thank you #2020-06-16 12:28:14 alexmiller Let me know how you make out #2020-06-16 12:28:53 robert-stuttaford interestingly this resolves fine on our CI instance, but not locally 🤔 on the CI, i see Skipping coordinate: with the same sha, presumably because it's already got it. of course, good old :local/root works locally. #2020-06-17 15:24:16 mokr Hi, I’m not sure where to start explaining to get the right context, but for short I’m trying to convert a somewhat large, working leiningen project to tools.deps. I’ve got the tools.deps + shadow-cljs dev experience working, but I’m struggling with the uberjar part.
The project is based on a Luminus template from 2017 so I haven’t done this part myself and I’m not an experienced Java developer either.
Right now I’m using this tool and guide (https://github.com/tonsky/uberdeps), but clj -Aclj -C:assets -e "(compile 'mtools.core)" (“clj” and “assets” are actual aliases) seems to just hang and nothing is created in the “classes” dir. At the terminal I see some output from a (println) in a middleware ns and some WARNING: An illegal reflective access ... and similar.
Any tips to help me along the way? E.g. how to get more verbose output from that compilation stage? #2020-06-17 15:40:28 hiredman Hanging when running compile usually indicates you have a top level form doing something that blocks/takes a long time #2020-06-17 15:42:54 mokr I’m quite confused here. Does it run the code when it compiles? I’m surprised to see that println output as well to be honest. #2020-06-17 15:43:15 hiredman It does #2020-06-17 15:43:58 hiredman Aot compilation is the same as regular code loading, it just saves the bytecode to disk as it generates it #2020-06-17 15:45:03 hiredman Likely your lein project was never aot compiled, because you would have experienced the same thing there #2020-06-17 15:46:49 mokr I’m quite confident that it was. Lein profile has :aot :all and after extracting the jar I see lots of .class files #2020-06-17 15:47:32 mokr So when it hangs it might be the Immutant web server doing its job, waiting for connections? #2020-06-17 15:48:17 hiredman Then you've made other changes than just switching from lein to deps.edn #2020-06-17 15:48:50 hiredman It could be, it depends how you are starting it #2020-06-17 15:49:27 hiredman I believe luminus uses mount to manage how and when things are started #2020-06-17 15:50:11 mokr I have certainly made quite a few changes, but not “that much” to that part of the code. #2020-06-17 15:50:16 mokr Yes, it uses mount. #2020-06-17 15:50:29 hiredman Past tense? #2020-06-17 15:50:35 mokr typo #2020-06-17 15:52:01 hiredman If you have something like (startmount) as a top level form, it will start mount while running compile #2020-06-17 15:52:14 mokr Due to shadow-cljs liking to have all it’s source paths in one listing, I’m having to move away from the “env” approach used in Luminus. That is: Luminus uses source path to include different versions of a namespace in dev vs prod. #2020-06-17 15:52:49 mokr I might have messed something up in that ongoing process. #2020-06-17 15:52:55 hiredman If you have a user.clj that starts mount when loaded #2020-06-17 16:00:17 mokr No user.clj, but mount is in core.clj that I call as main.
Mount has actually bitten me quite a few times. #2020-06-17 16:02:38 mokr Would have been way easier if my project was open source.
So, one way to get further is to start adding some printlns to my code, maybe? #2020-06-17 16:05:07 mokr Thanks, @hiredman at least I did learn something here. It’s time for dinner so I’ll be away for a while. #2020-06-18 19:06:27 markbastian Is there deps.edn analogue of the lein-environ plugin? I.e. a facility for setting an env map in a sidecar file (or your global deps.edn file) that is merged with the global env? #2020-06-18 19:12:45 alexmiller not to my knowledge #2020-06-18 19:13:12 markbastian Is it possible to set env vars in deps.edn or is that not really what it's meant for? #2020-06-18 19:13:32 alexmiller it's not possible right now #2020-06-18 19:13:50 alexmiller you can of course just set them on the call to clj using normal shell methods #2020-06-18 19:14:30 alexmiller I wouldn't rule out the possibility of adding something like that in the future #2020-06-18 19:14:49 markbastian Yeah, just trying to figure out the "right" way to set them when launching a REPL from Cursive. Using the Cursive config works fine for now. #2020-06-18 19:15:22 alexmiller in clj, one option is to create an alias that adds an :extra-paths and then load config via a resource #2020-06-18 19:15:55 alexmiller an alias can also set Java system properties #2020-06-18 19:16:03 alexmiller so those are two paths to have an alias add extra config #2020-06-18 19:17:26 markbastian That makes sense. "an alias can also set Java system properties" sounds promising. Thanks! #2020-06-18 19:17:57 alexmiller but be careful that you're not putting sensitive values in your deps.edn #2020-06-18 19:18:30 alexmiller you'd want to have a system property that point to a file (or env var) to read a sensitive value #2020-06-23 17:14:38 seancorfield I'd never heard of such a thing, even for Leiningen before. Looking at the code, I don't think it would be too hard to write a deps.edn-based tool that did the same thing if someone felt sufficient need... #2020-06-23 20:23:42 pmonks Yeah that was my thought too, and it helps a lot with license compliance (which has been a big deal most of the places I’ve worked). #2020-06-23 20:25:33 pmonks Is there a standard place in a deps.edn to put license information? Maven offers such a thing, though it’s not mandatory so not universally adopted (which is a shame). #2020-06-23 20:37:56 seancorfield There is nowhere in deps.edn to put such a thing. The repo might have pom.xml and you'd look in there, or look for a LICENSE file of some sort in the project root perhaps. #2020-06-23 20:41:05 seancorfield Be aware that clojure -Spom does not add <licenses> into the generated pom.xml -- I have it because I start my projects off with clj-new and the templates for pom.xml in app, lib, and template all have a default pom.xml file with an EPL 1.0 license listed in them. #2020-06-23 22:19:58 pmonks Right, and most of the tools that do this kind of thing “for real” rely on source repository analysis to determine licensing, rather than (potentially incorrect) declarations in build files like pom.xml.
The challenge in the Java ecosystem is that for dependencies often all you can reliably obtain is a pom.xml (and associated JAR(s)). That’s all lein-licenses uses, for example. #2020-06-23 22:22:14 pmonks This is (part of) the problem that the https://spdx.dev/ is trying to solve, and I’m a bit of an advocate for their efforts. They would get a huge boost if language ecosystems (such as Clojure’s) were to adopt SPDX for licensing information. #2020-06-23 22:24:10 pmonks While they’re better than some wild west ecosystems (cough Javascript cough), the JVM-based ecosystems are still a bit too cavalier in their approach to stating and disseminating licensing information, imho… #2020-06-23 21:10:46 drewverlee {}
Is a valid deps.edn right? #2020-06-23 21:11:43 seancorfield Yup. #2020-06-23 21:11:52 seancorfield Exactly the same as not having one. #2020-06-23 21:12:53 drewverlee What could cause two invocations of a the same clj command to result in different outcomes? i get a error building my claspath "dont know how to create an iseq from symbol" and everyone else gets the desired result. #2020-06-23 21:13:09 drewverlee something in my local maven repo? #2020-06-23 21:16:25 drewverlee clojure 1.10.1 #2020-06-23 21:18:34 drewverlee i blew away my m2 repo and got the same result. #2020-06-23 21:21:14 drewverlee that command as a missing curly bracket #2020-06-23 21:21:45 drewverlee nevermind, its fine #2020-06-23 21:24:09 drewverlee my argument order was wrong #2020-06-23 21:24:28 drewverlee i needed to pass -Srepro first then Sdeps #2020-06-23 21:38:48 seancorfield -Sdeps takes an argument, which is the {:deps ...} EDN. #2020-06-23 21:39:27 seancorfield So it's not so much that you need -Srepro first as that you need the EDN immediately following -Sdeps #2020-06-23 23:53:37 colliderwriter I have a repo with multiple related projects in it which i'm moving to tools.deps. Is there a way to include an external edn file so i can keep my aliases and common paths DRY? #2020-06-24 00:04:33 seancorfield @colliderwriter What we have done at work is to take advantage of the (undocumented?) CLJ_CONFIG environment variable to point the "user deps" path at a directory in our monorepo and have all the common aliases, overrides, etc in a deps.edn file in there -- including aliases for all the tooling needed for working across all the projects in our monorepo. #2020-06-24 00:07:08 seancorfield So our repo structure is:
clojure
|- versions (the common deps folder)
|- project-a
|- project-b
|- ...
and then we cd into a project to work and run clojure like this:
$ CLJ_CONFIG=../versions clojure -A:some:aliases and args
and it uses versions/deps.edn instead of a user-level deps.edn file plus the deps.edn from the project-x folder that we are running in. #2020-06-24 00:20:10 colliderwriter thanks for that. it will get the gears to mesh so i can move on today, but i foresee a problem with devops once i try to hook that up because bazel/rules_clojure won't know how to do that. This seems like a missing feature in tools.deps, especially because this is possible via lein-parent in old money. I guess i should go look at the I wish there were an issue list to see if anyone else has asked for this. #2020-06-24 00:35:16 colliderwriter I sure do. Thanks again. #2020-06-24 00:35:46 seancorfield Given that it is driven via an environment variable, and that could be automated via a simple "build" shell script in the repo, I don't see why it would cause problems for devops. #2020-06-24 00:36:42 seancorfield It has the benefit that you can control all tools directly within the repo since that "main" deps.edn file is under version control with everything else. #2020-06-24 00:41:42 colliderwriter I absolutely agree that the shared files are in the right place. I'm already using Bazel to do my builds and it is pretty opinionated. I'll have to look into my options there. #2020-06-24 01:43:49 hiredman I don't know bazel, but I suspect slot of those 'srcs' instead of being lists of source files should be directories containing source files #2020-06-24 01:47:14 hiredman clj, lein, boot, most clojure tooling has you specify what ends up being classpath roots (directories of classes or source code, and jar files via deps) not individual source files #2020-06-24 00:47:07 seancorfield Wow, that looks horribly restrictive. I'm sorry 😐 #2020-06-24 02:26:57 hiredman It is kind of a shame to have to explicitly list intra mono repo dependencies, given a closed mono repo world you can derive a project dependency graph from the namespace dependency graph #2020-06-25 09:19:35 danielcompton Does tools-deps support specifying version ranges? I couldn’t see anything about it. It looks like it supports it when reading a POM as a transitive dependency #2020-06-25 11:33:09 dominicm I'm pretty sure it doesn't #2020-06-25 11:33:11 dominicm By design #2020-06-25 11:37:53 dominicm It just gets the highest version in the range #2020-06-25 14:05:20 murtaza52 I have put my aliases in the ~/.clojure/deps.edn file (I am suing the dot file from seancorfield), but when I try to call clj -A:new app test/one , I get the following error -
Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
app (No such file or directory)
Full report at:
/tmp/clojure-3392454598929432067.edn
It seems it is not able to find the alias, because when I inline the alias and call the command it works. Why is it not recognizing the ~/.clojure/deps.edn file ? #2020-06-25 15:34:12 seancorfield @murtaza52 Did you also get this WARNINg about that error message?
WARNING: Specified aliases are undeclared: [:new]
What does clojure -Sdescribe show for you? #2020-06-25 15:36:44 murtaza52 ~ >>> clojure -Sdescribe [1]
{:version "1.10.1.536"
:config-files ["/usr/local/lib/clojure/deps.edn" "/home/murtaza52/.config/clojure/deps.edn" ]
:config-user "/home/murtaza52/.config/clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/usr/local/lib/clojure"
:config-dir "/home/murtaza52/.config/clojure"
:cache-dir "/home/murtaza52/.config/clojure/.cpcache"
:force false
:repro false
:resolve-aliases ""
:classpath-aliases ""
:jvm-aliases ""
:main-aliases ""
:all-aliases ""} #2020-06-25 15:37:14 murtaza52 @seancorfield No I did not get that warning #2020-06-25 15:39:55 murtaza52 I moved the file /.clojure/deps.edn to /.config/clojure/deps.edn, but I am still getting the same output as above. #2020-06-25 15:46:46 seancorfield How did you install the Clojure CLI tools? I've never seen it try to use ~/.config/clojure as the user-level location... #2020-06-25 15:47:38 seancorfield What O/S is this on? What is your HOME environment variable? (`echo $HOME`) #2020-06-25 15:48:08 murtaza52 ~ >>> echo $HOME
/home/murtaza52 #2020-06-25 15:48:17 borkdude this is caused by xdg-config stuff on linux right #2020-06-25 15:48:23 murtaza52 This is manjaro linux #2020-06-25 15:51:00 murtaza52 sorry I m still lost, how should I fix it ? where does that deps.clj go ? #2020-06-25 15:51:21 seancorfield It's a separate tool. He's just showing that the XDG thing is a problem. #2020-06-25 15:51:49 murtaza52 this is a new install of manjaro without a DE, I had installed it before with DE and it worked fine. #2020-06-25 15:51:54 borkdude yeah, I should point to the bash script, this is a clojure port of it #2020-06-25 15:52:06 borkdude can you do echo $XDG_CONFIG_HOME? #2020-06-25 15:52:11 murtaza52 this one just bspwm as the tiling manager #2020-06-25 15:52:52 murtaza52 ~ >>> echo $XDG_CONFIG_HOME
/home/murtaza52/.config #2020-06-25 15:52:54 borkdude and then try with the -Sforce with your changes in place #2020-06-25 15:53:09 borkdude yes, so it will look for /home/.../.config/clojure/deps.edn #2020-06-25 15:54:15 murtaza52 but in already has a dir called /.config/clojure , should I replace it with /.config/.clojure ? #2020-06-25 15:55:14 borkdude no sorry, without the dot, it was a typo on my behalf #2020-06-25 15:55:45 borkdude and then re-try your action with -Sforce #2020-06-25 15:56:07 murtaza52 what does sforce do ? yes it worked ! #2020-06-25 15:56:18 borkdude it ignores a cached file #2020-06-25 15:56:55 murtaza52 cool thanks ! #2020-06-25 15:58:15 seancorfield New topic: tools.deps.alpha ignores :scope in a coordinate, yes? That's a Maven thing that Lein/Boot support but is not relevant for deps.edn...? #2020-06-25 16:01:39 seancorfield Hahaha... well, someone opened a bug against depstar about it putting :scope "provided" dependencies into the JAR and when I created a quick test to check the classpath clojure -Spath in a new project with a :scope "provided" dependency, yes, it's on the classpath so this is t.d.a behavior. #2020-06-25 17:23:16 souenzzo hello. I'm trying to use git deps with aws codecommit, inside a ec2 instance but I get
Error building classpath. : Authentication is required but no CredentialsProvider has been registered
but when I (inside the ec2 instance) git clone it works
Anyone else using tools_deps + codecommit?
(stacktrace on comment) #2020-06-25 17:23:26 souenzzo Cloning:
Error building classpath. : Authentication is required but no CredentialsProvider has been registered
org.eclipse.jgit.api.errors.TransportException: : Authentication is required but no CredentialsProvider has been registered
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:248)
at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:306)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:200)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:89)
at clojure.tools.gitlibs.impl$call_with_auth.invokeStatic(impl.clj:50)
at clojure.tools.gitlibs.impl$call_with_auth.invoke(impl.clj:41)
at clojure.tools.gitlibs.impl$git_clone_bare.invokeStatic(impl.clj:71)
at clojure.tools.gitlibs.impl$git_clone_bare.invoke(impl.clj:68)
at clojure.tools.gitlibs.impl$ensure_git_dir.invokeStatic(impl.clj:110)
at clojure.tools.gitlibs.impl$ensure_git_dir.invoke(impl.clj:100)
at clojure.tools.gitlibs$resolve.invokeStatic(gitlibs.clj:33)
at clojure.tools.gitlibs$resolve.invoke(gitlibs.clj:29)
at clojure.tools.gitlibs$procure.invokeStatic(gitlibs.clj:47)
at clojure.tools.gitlibs$procure.invoke(gitlibs.clj:41)
at clojure.tools.deps.alpha.extensions.git$eval999$fn__1001.invoke(git.clj:41)
at clojure.lang.MultiFn.invoke(MultiFn.java:239)
at clojure.tools.deps.alpha$expand_deps.invokeStatic(alpha.clj:189)
at clojure.tools.deps.alpha$expand_deps.invoke(alpha.clj:169)
at clojure.tools.deps.alpha$resolve_deps.invokeStatic(alpha.clj:237)
at clojure.tools.deps.alpha$resolve_deps.invoke(alpha.clj:217)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invokeStatic(make_classpath2.clj:55)
at clojure.tools.deps.alpha.script.make_classpath2$create_classpath.invoke(make_classpath2.clj:47)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invokeStatic(make_classpath2.clj:81)
at clojure.tools.deps.alpha.script.make_classpath2$run_core.invoke(make_classpath2.clj:72)
at clojure.tools.deps.alpha.script.make_classpath2$run.invokeStatic(make_classpath2.clj:101)
at clojure.tools.deps.alpha.script.make_classpath2$run.invoke(make_classpath2.clj:95)
at clojure.tools.deps.alpha.script.make_classpath2$_main.invokeStatic(make_classpath2.clj:146)
at clojure.tools.deps.alpha.script.make_classpath2$_main.doInvoke(make_classpath2.clj:118)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.core$apply.invokeStatic(core.clj:665)
at clojure.main$main_opt.invokeStatic(main.clj:514)
at clojure.main$main_opt.invoke(main.clj:510)
at clojure.main$main.invokeStatic(main.clj:664)
at clojure.main$main.doInvoke(main.clj:616)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.lang.Var.applyTo(Var.java:705)
at clojure.main.main(main.java:40)
Caused by: org.eclipse.jgit.errors.TransportException: : Authentication is required but no CredentialsProvider has been registered
at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:530)
at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:345)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:137)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:123)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1269)
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:237)
... 37 more #2020-06-25 17:32:59 hiredman jgit may not support whatever authentication method git is using there #2020-06-25 17:44:16 ghadi you need to use ssh auth, not https #2020-06-25 18:17:44 ghadi yes #2020-06-25 18:17:50 ghadi and put the key in an ssh-agent #2020-06-25 20:19:26 souenzzo @U050ECB92 that't not aplicable once I'm trying to use LaunchConfig #2020-06-25 19:26:45 murtaza52 If I am using a template to generate a project, and the template has changed, will just using -Sforce force the new template to be retrieved ? #2020-06-25 19:28:53 seancorfield No. The template is only used when the project is first created. #2020-06-25 19:29:17 seancorfield After that, it's just "your project" and no longer related to the template. #2020-06-25 19:53:16 murtaza52 nope want to create a new project with the template, and the template was recently updated, how do I ensure that the updated template from clojars will be pulled for the new project ? #2020-06-25 19:54:56 seancorfield By default, clj-new always uses the most recent stable version of a template. #2020-06-25 19:55:56 seancorfield Per the clj-new documentation (README), you can override that behavior:
-S or --snapshot -- look for -SNAPSHOT version of the template (not just a release version)
-v or --verbose -- enable debugging -- be verbose! -vv and -vvv are increasingly verbose
-V or --version, followed by a version -- use this specific version of the template
#2020-06-25 19:58:21 murtaza52 cool thanks #2020-06-25 22:05:23 seancorfield @danvingo It will use whatever version t.d.a. would select if it were a dependency with a "RELEASE" version. #2020-06-25 22:05:35 seancorfield What is the template you're trying to work with? #2020-06-25 22:06:48 dvingo I wanted to use dates as the versions... I'm just learning about the wonderful world of maven version semantics #2020-06-25 22:07:48 seancorfield I posted the options above that would let you do that. #2020-06-25 22:08:56 seancorfield When I tried it -- having never looked at that template before -- here's what I got:
(! 709)-> clj -A:new dv.fulcro-template dvingo/example -vvv
Downloading: dv/fulcro-template/clj-template/maven-metadata.xml from clojars
Output from locating template:
Downloading: dv/fulcro-template/clj-template/2020-06-25-4/clj-template-2020-06-25-4.pom from clojars
Downloading: dv/fulcro-template/clj-template/2020-06-25-4/clj-template-2020-06-25-4.jar from clojars
Generating fresh 'clj new' dv.fulcro-template project.
#2020-06-25 22:10:11 dvingo ah thank you! sorry I overlooked that #2020-06-25 22:10:59 seancorfield And here's how to pull down specific versions:
(! 714)-> clj -A:new dv.fulcro-template dvingo/example -V 2020-06-25-4
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:09:24
(sean)-(jobs:0)-(~/clojure)
(! 715)-> clj -A:new dv.fulcro-template dvingo/example2 -V 2020.06.3
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:09:55
(sean)-(jobs:0)-(~/clojure)
(! 716)-> clj -A:new dv.fulcro-template dvingo/example3 -V 2020-06-25-3
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:10:29
(sean)-(jobs:0)-(~/clojure)
(! 717)-> clj -A:new dv.fulcro-template dvingo/example1 -V 1.0.0
Generating fresh 'clj new' dv.fulcro-template project.
Thu Jun 25 15:10:41
(sean)-(jobs:0)-(~/clojure)
(! 718)->
#2020-06-25 22:13:10 dvingo hmm, I'm on a machine that has prior versions of the template in .m2 and I'm not seeing what you're seeing:
$ clj -A:new dv.fulcro-template dvingo/example -vvv
Output from locating template:
Generating fresh 'clj new' dv.fulcro-template project.
#2020-06-25 22:14:19 seancorfield Well, you won't see any output if it doesn't have to download anything. #2020-06-25 22:15:02 dvingo It at some point stopped downloading the newest versions:
Thu Jun 25, 2020 18:05:15:~/.m2/repository/dv/fulcro-template/clj-template ()
$ l
total 24
drwxr-xr-x 9 default staff 288B Jun 25 18:12 ./
drwxr-xr-x 5 default staff 160B Jun 3 13:01 ../
drwxr-xr-x 7 default staff 224B Jun 23 21:47 2020-06-23-2/
drwxr-xr-x 7 default staff 224B Jun 25 13:31 2020-06-24/
drwxr-xr-x 7 default staff 224B Jun 25 13:56 2020-06-25/
drwxr-xr-x 7 default staff 224B Jun 25 14:05 2020-06-25-2/
-rw-r--r-- 1 default staff 918B Jun 25 14:05 maven-metadata-clojars.xml
-rw-r--r-- 1 default staff 40B Jun 25 14:05 maven-metadata-clojars.xml.sha1
-rw-r--r-- 1 default staff 280B Jun 25 18:12 resolver-status.properties
#2020-06-25 22:16:15 seancorfield I think t.d.a. only checks once per day for releases. #2020-06-25 22:16:36 dvingo ahhh, ok #2020-06-25 22:17:12 seancorfield It's very unusual to have multiple releases on the same day. Even with snapshots. #2020-06-25 22:18:07 seancorfield If you blow away ~/.m2/repository/dv/fulcro-template/clj-template then it should download whatever is the latest version. #2020-06-25 22:18:10 dvingo yea that makes sense to save time for the usual case. I wasn't aware of how the interaction with maven occurred - thanks for the help!
deleting the dir from the .m2 repo made it pull the latest #2020-06-25 22:18:21 seancorfield Yup, that 🙂 #2020-06-25 22:19:02 seancorfield Like I said above, you can always use the -V / --version option to download a specific version. #2020-06-25 22:19:30 dvingo right -thanks again! #2020-06-25 22:22:12 seancorfield You may be able to use a version range like this -V '[0,)' but, again, Aether (the Maven-stuff underneath t.d.a.) may only check once a day. #2020-06-25 22:24:39 seancorfield (FWIW, I downloaded 2020-06-25-2 per your directory listing above, then used [0,) and it did pull down 2020-06-25-4... @danvingo #2020-06-25 22:25:13 seancorfield So it may be that the range checks every time? (maybe a Maven expert can confirm/deny that?) #2020-06-25 22:26:36 dvingo gotcha - my main use case is template users and not wanting to have them specify versions
i'll just use ComparableVersion at the repl before sending a new version 🙂 #2020-06-25 22:26:51 seancorfield (hard to tell -- having pulled down the -2 version, running it without a version still pulled down -4 for me, based on RELEASE instead) #2020-06-25 22:27:20 seancorfield If you're not doing more than one release a day, I suspect you won't have a problem 🙂 #2020-06-25 22:28:30 dvingo perfect, thanks! #2020-06-25 22:36:11 seancorfield One final thing @danvingo if you use the -S option, it uses a version range of (0.0.0,) #2020-06-25 22:38:59 dvingo ahh good to know.
I think the convention I wanted to use is working - 2020-06-25 compares -1 to 2020-06-25-1 and 2020-06-25-1 comes before 2020-06-25-2
so looks like the issue was maven not being checked by t.d.a. and not version sorting #2020-06-25 22:40:45 seancorfield This is partly why I've moved to MAJOR.MINOR.COMMITS so I don't have to worry about that. COMMITS is always increasing. #2020-06-25 22:43:19 dvingo makes sense - I just want to do my part to help kill semantic(less) versioning (I know you're not following that, per the note in the readme, but it still follows the syntax) #2020-06-25 22:48:31 seancorfield Agreed. There are plenty of ways to do it. YYYY-MM-DD.SEQ is as good as any -- and there are plenty of libraries out there, even in the Java world, that use date/time versions instead of semantic(less) versioning (I like that term... Imma gonna stealz it!) #2020-06-26 01:08:47 dvingo awesome, that's good to hear, and yes pls do! #2020-06-26 17:06:50 pmonks Slightly OT, but does anyone have any good links to further reading on why semantic versioning is problematic? (and yes, I’ve watched the video of Mr Hickey’s presentation on the topic) #2020-06-26 17:51:57 pmonks Thanks! #2020-06-26 17:58:54 pmonks So criticism is great, but what’s the alternative? #2020-06-26 17:59:05 pmonks (let me move this to #off-topic) #2020-06-27 20:33:18 practicalli I published a page on using the very unofficial tools.deps hot-loading of dependencies. Be aware that this is very unofficial and no guarantee it will be included in anything official. Credit to Sean Corfield for the alias configuration. I've shown an example of using hot loading with the rebel REPL. https://practicalli.github.io/clojure/alternative-tools/clojure-tools/hot-load-dependencies.html
Again, this is very unofficial, so use with caution and dont rely on it being available. #2020-06-27 20:45:29 alexmiller Given that this is an unreleased branch of an alpha lib and we’ve already determined the api is going to change, can you make it real obvious on that page that this is not “official” stuff? #2020-06-27 20:46:27 alexmiller There is a wip add-lib2 branch out there but it is actually a branch off another wip branch (which has now moved forward and not been merged back) #2020-06-27 20:48:04 alexmiller It’s entirely possible for that existing add-lib branch to become broken at any time (as master drifts), and I actually expect that to happen soon #2020-06-27 20:54:21 seancorfield @alexmiller Understood. I rely on it quite a bit but I'm happy to switch to any other working version, if I can just prevail on you to point me at such a thing if the SHA I'm depending on right now breaks (and I'm happy to use Clojure master for dev, if that's what I have to switch to) #2020-06-27 21:02:10 alexmiller There may be a period of time where there is no working add-lib branch #2020-06-27 21:03:29 seancorfield I haz a sad 😢 #2020-06-27 21:03:46 seancorfield Fair enough. I can survive for a "period of time" without it I guess. #2020-06-27 21:05:06 alexmiller I will try to get it remerged, just won’t be my priority so may take a few days as the tools.build stuff moves forward #2020-06-27 21:05:24 seancorfield Cool. And I'm happy to test that out too, as you know 🙂 #2020-06-27 20:55:57 seancorfield ^ @jr0cket I would have tried to dissuade you from writing this up, if you had asked me ahead of time. Even tho' it's in my public dot-clojure file and I use it day-in, day-out, I wouldn't want to see that baked into the public documentation base around Clojure... #2020-06-27 20:58:26 practicalli I did add a caveat to that effect in the deps.edn file, however, I should have put that same warning on the page too. I will update asap #2020-06-27 21:04:41 seancorfield @jr0cket This is why I've held off doing a YouTube video showing how I use add-lib on that t.d.a branch -- I don't think people should be encouraged to use it when it's likely to go away/break/change. #2020-06-27 21:09:44 practicalli @alexmiller I've updated the page with a large up front warning. If you prefer I remove the page entirely, I am happy to do so.
I appreciate there are other approaches and I should probably work on documentation for them in hindsight, its just someone was asking me how to specifically do this hot reloading. #2020-06-27 21:16:40 alexmiller It’s a public thing, so I don’t care if people are aware of it, just want to be sure it’s experimental nature is clear #2020-06-28 12:33:07 practicalli I've moved the page on add-lib to a reference section of the book, as an alternative and very much experimental approach. Apologies for not thinking about this better. #2020-06-28 10:32:46 borkdude What does this mean?
Exception in thread "main" clojure.lang.ExceptionInfo: Manifest type not detected when finding deps for babashka.curl/babashka.curl in coordinate {:git/url "", :sha "2dc7f53271de3c2edc3e1474a000b9dfa7324eaf"} {:lib babashka.curl/babashka.curl, :coord {:git/url "", :sha "2dc7f53271de3c2edc3e1474a000b9dfa7324eaf"}}, compiling:(/tmp/prog.clj:3:1) #2020-06-28 10:32:57 borkdude as a result of:
(require '[clojure.tools.deps.alpha :as deps])
(-> {:deps '{babashka.curl
{:git/url ""
:sha "2dc7f53271de3c2edc3e1474a000b9dfa7324eaf"}}
:mvn/repos {"central" {:url ""}
"clojars" {:url ""}}}
(deps/resolve-deps nil)
(deps/make-classpath nil nil)) #2020-06-28 10:40:44 borkdude I get the same result when using this coordinate with -Sdeps from the command line #2020-06-28 12:42:46 alexmiller this should not affect anything #2020-06-28 13:47:05 lread oh, looks like I misdiagnosed! thanks for the info! #2020-06-28 13:48:55 alexmiller it's just map merges and stuff in the code, so I would not expect it to violate any Clojure-ish expectations here #2020-06-28 11:02:44 borkdude That appears to be it. Weird. #2020-06-28 11:03:06 borkdude Thanks! #2020-06-28 11:05:06 lread You are most welcome, by chance I hit that one yesterday. #2020-06-28 12:43:02 alexmiller do you have a repro of that? #2020-06-28 11:26:38 borkdude @lee Hmm, now it's coming back:
$ clj -Sdeps "{:deps {babashka.curl
{:git/url \"\"
:sha \"cbf8429addac1d2f71ac8ddfb847657223fdf0f7\"}}}" -Spath
Error building classpath. Manifest type not detected when finding deps for babashka.curl/babashka.curl in coordinate {:git/url "", :sha "cbf8429addac1d2f71ac8ddfb847657223fdf0f7"}
#2020-06-28 11:28:23 borkdude it did work for:
$ clj -Sdeps "{:deps {babashka.curl
{:git/url \"\"
:sha \"1dc0f61d7baabee1f009543d60bfb79ee4a7101c\"}}}" -Spath #2020-06-28 11:29:30 borkdude while the commit cbf8 didn't do anything relevant compared to 1dc0with the deps.edn file.... #2020-06-28 11:31:15 borkdude Adding :deps/manifest :deps seems to fix it. #2020-06-28 11:33:18 borkdude Added this to the issue #2020-06-28 11:53:18 borkdude I'm also getting this for a local/root dep:
$ clj -Sdeps "{:deps {org.clojure/tda {:local/root \"/Users/borkdude/Dropbox/dev/temp/tools.deps.alpha\" }}}" -Spath
Error building classpath. Manifest type not detected when finding deps for org.clojure/tda in coordinate #:local{:root "/Users/borkdude/Dropbox/dev/temp/tools.deps.alpha"} #2020-06-28 12:03:41 alexmiller Are they reproducible with -Sforce #2020-06-28 12:04:47 borkdude same result #2020-06-28 12:05:20 borkdude /tmp $ clj -Sdeps "{:deps {babashka.curl
{:git/url \"\"
:sha \"cbf8429addac1d2f71ac8ddfb847657223fdf0f7\"}}}" -Spath -Sforce -Srepro
Error building classpath. Manifest type not detected when finding deps for babashka.curl/babashka.curl in coordinate {:git/url "", :sha "cbf8429addac1d2f71ac8ddfb847657223fdf0f7"} #2020-06-28 12:38:05 alexmiller @borkdude I can't repro any of your failing cases #2020-06-28 12:39:34 borkdude Hah. Worked after a rm -rf ~/.gitlibs #2020-06-28 12:39:44 borkdude Not for
clj -Sdeps "{:deps {org.clojure/tda {:local/root \"/Users/borkdude/Dropbox/dev/temp/tools.deps.alpha\" }}}" -Spath
though #2020-06-28 12:41:04 borkdude That's probably another problem. It does work with a vanilla checkout of tools.deps:
clj -Sdeps "{:deps {org.clojure/tda {:local/root \"tools.deps.alpha\" }}}" -Spath #2020-06-28 12:41:36 borkdude I'll close the issue. #2020-06-28 12:44:12 alexmiller tools.deps.alpha has a couple build steps (java compilation, resource replacement) so it's not a good example lib for use as local/git dep #2020-06-28 12:50:44 alexmiller btw, in the near future, using unqualified lib symbols is going to be deprecated and will complain, so "babashka.curl" should be changed to "babashka.curl/babashka.curl" etc #2020-06-28 12:51:48 alexmiller (and this is a step towards actually removing support for that) #2020-06-28 13:24:09 borkdude @alexmiller I know. I had already compiled the java and added target/classes to the paths #2020-06-28 13:24:20 borkdude But still didn’t work probably due to the resource replacement #2020-06-28 13:26:18 borkdude I’m trying to get the native tools deps working using Ghadi’s git shell branch of the git libs library, but this stuff got me distracted for now #2020-06-28 13:30:06 borkdude Should probably push the compiled assets to Clojars and work with that instead of tools deps deps #2020-06-28 13:36:33 borkdude Btw, the url of a git dep is already a unique identifier. Why should a user have to make up some fully symbol keyword in the first place? #2020-06-28 13:41:17 alexmiller well you can just mvn clean install and use that local snapshot artifact too - sometimes that's eaesiest for maven built things #2020-06-28 13:47:57 alexmiller re unique id, for the exact same reasons namespaces are useful for global differentiation, group-ids are a good idea for qualified names. using same symbol group+artifact was always a limited idea. lein/clojars made it too easy and common. we're not going to perpetuate making it implicit. #2020-06-28 13:52:46 borkdude But why have a name for a git dep at all? #2020-06-28 13:54:45 alexmiller because names are the identity of the lib, which may exist at different coordinates and you want to avoid putting the "same" lib on the classpath from multiple locations #2020-06-28 13:55:55 alexmiller (admittedly still some work to do on cross-coordinate type version selection, but the hooks are there) #2020-06-28 13:57:17 borkdude Good point. The only use case for doing that would be when you would mount different dirs from the same git dep at different SHAs, which seems far fetched. Still if you would use the URL as the identifier you would be able to detect conflicts more easily since different people are going to use different qualified syms for the same lib? #2020-06-29 08:07:50 rickmoynihan Not that far fetched as we actually do this in one or two places.
One of our apps supports different customer configurations via t.d.a aliases, and there is essentially a :customer/all alias that pulls together a combined classpath for all customers, for working at the REPL (which lets you switch between customers).
One of the deps in that alias contains fixture data for all customers with a :deps/root for each customer, and it’s possible for the sha’s for customers to drift slightly from each other (even though they’re all in the same repo). So this t.d.a feature is very handy, as otherwise we’d need a fixture data repo per customer, which would make updating fixture harder across customers harder.
Essentially it just means the artifact id in this becomes git url + deps-root. It makes a lot of sense when you use :deps/root.
Normally they all reference the same SHA, but sometimes it’s pragmatic to apply changes to one customer first, before rolling out wider. This can also happen when you want a branch to built repeatably by C.I. (before you’ve applied the change across all customers). #2020-06-29 08:17:24 rickmoynihan We then coin a local dep-id of something like: swirrl/fixtures.customer-name #2020-06-29 08:19:53 borkdude Yeah, that does make sense. #2020-06-29 08:21:25 rickmoynihan It’s essentially just a lighter weight way to manage separate dependencies that you often make cross-cutting changes over. #2020-06-29 08:24:11 rickmoynihan Though I do suspect t.d.a might have some bugs resolving these sometimes — I’ve seen a few cases where when fetching these deps it seems to either blow up or mix up the commits, e.g. it would resolve customer-b to customer-a’s sha.
I suspect it might be related to the parallel resolution stuff. cc @alexmiller #2020-06-29 08:24:28 rickmoynihan It’s quite hard to reproduce though #2020-06-29 13:09:11 alexmiller yes, I am almost certain that it's parallel resolution of git deps #2020-06-28 13:58:39 alexmiller we do not currently do that level of detection but it is something I would like to do #2020-06-28 13:59:06 alexmiller ultimately the library has to determine the identifier by which it should be known #2020-06-28 14:01:19 alexmiller the organization should be something you own or register, like domain name, trademark. third party sites like github can also confer identity - we recommend in https://clojure.org/guides/deps_and_cli to combine them by creating a group-id like github-borkdude #2020-06-28 14:01:19 borkdude Which the URL is already an instance of #2020-06-28 14:04:15 alexmiller url is really a combination of site (github), org, and repo. github-org/repo combines those #2020-06-28 14:04:21 alexmiller but it does not account for forks #2020-06-28 14:04:51 borkdude Interesting 🤔 #2020-06-28 14:05:02 alexmiller to use a fork, you need to use the same lib, but different url, and then they diverge #2020-06-28 14:05:50 alexmiller your goal with using a fork really is to replace (identity) the lib #2020-06-28 14:06:12 alexmiller but then you have interesting questions of ownership over the lib identity #2020-06-28 14:06:13 borkdude If I want to maintain a fork as a real fork, and not just for PRs, I would not use the original identifier anyway? #2020-06-28 14:06:33 alexmiller well that's the question #2020-06-28 14:06:45 borkdude So then the URL still is the most identifying thing for it #2020-06-28 14:07:09 alexmiller if you didn't use the original, then dep resolution will give you both libs if you have a transitive dep using the original #2020-06-28 14:07:38 alexmiller and in that case, it would be good to have deps tell you that you've done this #2020-06-28 14:07:43 borkdude It’s the same in maven world right now #2020-06-28 14:07:47 alexmiller true #2020-06-28 14:08:13 alexmiller although probably much harder to automatically detect #2020-06-28 14:09:11 alexmiller (note that git urls are also not unique - there are multiple url formats to refer to the same repo, but I think that's a solvable problem) #2020-06-28 16:27:14 borkdude I have got something compiling now with tda + /cdn-cgi/l/email-protection branch
One issue I stumbled upon:
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved type during parsing: org.apache.maven.project.ReactorModelPool. To diagnose the issue you can use the --allow-incomplete-classpath option. The missing type is then reported at run time when it is accessed the first time.
Detailed message:
Trace:
at parsing clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(pom.clj:51)
Call path from entry point to clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(Object):
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invokeStatic(pom.clj:41)
at clojure.tools.deps.alpha.extensions.pom$model_resolver.invoke(pom.clj:41)
at clojure.core.proxy$clojure.lang.APersistentMap$ff19274a.hashCode(Unknown Source)
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at java.util.Properties.getProperty(Properties.java:1125)
at com.oracle.svm.core.jdk.SystemPropertiesSupport.getProperty(SystemPropertiesSupport.java:144)
at com.oracle.svm.core.jdk.Target_java_lang_System.getProperty(JavaLangSubstitutions.java:345)
at com.oracle.svm.jni.JNIJavaCallWrappers.jniInvoke_VA_LIST:Ljava_lang_System_2_0002egetProperty_00028Ljava_lang_String_2_00029Ljava_lang_String_2(generated:0)
I now use --allow-incomplete-classpath just to see how far it gets.
I now get a NPE in util/maven.clj on the line local-repo-mgr (.newLocalRepositoryManager system session local-repo) where system appears to be null. Might be related to the --allow-incomplete-classpath but I'll keep digging. #2020-06-28 16:29:39 alexmiller It is almost certainly related #2020-06-28 16:30:36 alexmiller There is a similar thing like this logged where an error occurs while setting up the service locator #2020-06-28 16:38:58 borkdude So ditching jgit for shell git works. When are you going to ditch org.apache.maven libs? 😉 #2020-06-28 16:47:19 borkdude I wonder what's the issue with this class #2020-06-28 18:49:01 ghadi before using a solution like shelling out to git, I think it’s wise to re-examine the problem #2020-06-28 18:49:28 ghadi I know I sound like a broken record #2020-06-28 18:56:04 borkdude Alex pointed me at that shell-git branch when I told him jgit didn't work with GraalVM. Just examining + listing all the problems. Shelling out to git allows me to explore the next problem without being stuck on it. There might not be only one "the problem", there might be many. #2020-06-28 19:07:11 ghadi what part of jgit didn't work? #2020-06-28 19:09:27 borkdude so if there's need, it can be made to work, I'm pretty sure #2020-06-29 08:49:08 holyjak Hello! Could somebody be so kind and explain to me why clj is trying to download a dependency from Maven Central when I told it to you our company repository (I assume M.C. is always used as a back up; but it doesn't seem it has even tried?!:
$ cat deps.edn
{:mvn/repos {"telia-aws" {:url ""}}
:deps {com.oracle.jdbc/ojdbc8 {:mvn/version "12.2.0.1"}}}
$ deps.exe -Sverbose
deps.clj version = 0.0.8
...
Refreshing classpath
Error building classpath. Could not find artifact com.oracle.jdbc:ojdbc8:jar:12.2.0.1 in central ()
🙏 Thank you! #2020-06-29 08:51:13 borkdude I think that error message just picks a repo and doesn't list all of them #2020-06-29 10:45:33 holyjak It seems you were right! The problem turned out to be that my URL was wrong, it was missing a part of the path. The -Sverbose option was quite useless for troubleshooting this... :-( #2020-06-29 11:13:58 holyjak Any idea why is clj trying to download a snapshot .jar from Central (where it isn't) when I have it in the local .m2 repo? Do I need to tell it to use the local repo??
> Error building classpath. Could not find artifact no.telia:clj-common:jar:0.1.0-20200507.122109-9 in central (https://repo1.maven.org/maven2/)
ls ~/.m2/repository/no/telia/clj-common/0.1.0-SNAPSHOT/clj-common-0.1.0-20200507.122109-9.jar
/Users/me/.m2/repository/no/telia/clj-common/0.1.0-SNAPSHOT/clj-common-0.1.0-20200507.122109-9.jar
The dependency is declared as:
no.telia/clj-common {:mvn/version "0.1.0-20200507.122109-9"} ;; a particular SNAPSHOT
#2020-06-29 11:16:23 borkdude SNAPSHOT versions are refreshed from external repositories at least once a day (by default, you can configure this in .m2 settings I believe) #2020-06-29 11:16:47 borkdude it's maven behavior #2020-06-29 11:20:25 holyjak Ah, ok. And if it fails to refresh, it just crashes without using the old version? #2020-06-29 13:25:20 alexmiller deps doesn't currently expose this setting #2020-06-29 15:51:31 holyjak So it doesn't take all of settings.xml into account, only parts of it (ie auth)? #2020-06-29 15:52:16 alexmiller it does not consider that setting, it does take more than just auth though #2020-06-29 15:53:52 alexmiller specifically, also supports mirrors and proxies #2020-06-29 15:53:59 alexmiller and http properties #2020-06-29 11:32:42 holyjak Hm, it seems that clj simply ignores my local .m2/repository. Now it is trying to download com.oracle.jdbc:ucp:pom:12.2.0.1 even though ~/.m2/repository/com/oracle/jdbc/ucp/12.2.0.1/ucp-12.2.0.1.jar exists...:
$ cat deps.edn
{:deps {com.oracle.jdbc/ucp {:mvn/version "12.2.0.1"}}}
$ clj
Error building classpath. Could not find artifact com.oracle.jdbc:ucp:jar:12.2.0.1 in central ()
$ ls -lh ~/.m2/repository/com/oracle/jdbc/ucp/12.2.0.1/ucp-12.2.0.1.jar
-rw-r--r-- 1 holyjak staff 1.0M Apr 2 13:55 /Users/holyjak/.m2/repository/com/oracle/jdbc/ucp/12.2.0.1/ucp-12.2.0.1.jar
Why?! #2020-06-29 13:23:15 alexmiller did you put that jar manually in your .m2? #2020-06-29 13:24:30 alexmiller if so, you may be missing some metadata in your local cache that's triggering a lookup #2020-06-29 15:52:23 holyjak No, via mvn install-file . And it used to work... #2020-06-29 15:56:12 alexmiller based on what I see here, I would expect it to find and use the jar in your local cache #2020-06-29 15:56:32 alexmiller you might want to clj -Sforce to make sure you don't have a cached classpath or something #2020-06-29 15:57:23 alexmiller I don't think that will matter since it seems to be building the classpath when it encounters the error, but just a good habit when diagnosing clj stuff #2020-06-29 20:43:22 holyjak Ok, thank you! #2020-06-30 11:22:05 holyjak FYI my problem is fixed now. I believe something was wrong with my .m2/repository #2020-06-30 12:01:32 alexmiller Cool #2020-06-30 15:46:41 unbalanced Heya @alexmiller I'm reading through tools.deps and you're using some pretty advanced kung fu in there. I don't suppose in all your ample free time you've given any talks or write-ups on a deep dive there have you? I'm trying to write some clojurescript CLI tools for working with deps and trying to port some of the stuff. Also, I tried running with CIDER but nREPL on the deps ran into the predictable errors for including java classpaths -- I'm curious what your REPL technique is for developing tools.deps.alpha? #2020-06-30 16:44:08 alexmiller The last conj talk is probably the most detailed info about the impl. What part are you having trouble with? Re repl, I just open it in Cursive as a Maven project #2020-06-30 15:47:24 borkdude @goomba There are a couple of tools.deps talks, they are listed in the repo #2020-06-30 15:48:55 unbalanced I was there for the last one. I think I'm looking for more "how to contribute" style info #2020-06-30 15:49:19 unbalanced 😄 reading seems like a good start #2020-06-30 15:49:22 borkdude @goomba ClojureScript already has cljs.main which works great in tandem with tools.deps? #2020-06-30 15:50:00 unbalanced yeah of course. I'm not trying to replicate that functionality -- I'm working on something more like "pip for deps" #2020-06-30 15:50:16 unbalanced I'm lazy and I don't like editing my deps file by hand #2020-06-30 15:50:22 unbalanced also I can never remember version numbers #2020-06-30 15:50:47 unbalanced I've got it working for clojars but there's a bunch of stuff on git and maven and I don't know how to work with those yet #2020-06-30 15:51:02 borkdude interesting, anything public yet? #2020-06-30 15:51:31 unbalanced not yet because it's still a mess but I'll send you some stuff on what I've got #2020-06-30 15:53:26 dpsutton there is a project out there that will search and add things to your deps files #2020-06-30 15:57:46 unbalanced Ahh that's great. Reason I'm going the cljs route is just b/c of startup time etc #2020-06-30 15:58:00 unbalanced and for personal education, I suppose #2020-06-30 15:58:12 borkdude @goomba if startup time is important, you can also use GraalVM and/or babashka #2020-06-30 15:58:29 unbalanced I need to learn to use both of those, for sure #2020-06-30 15:58:32 borkdude (all good options, including what you're doing now of course!) #2020-06-30 15:58:36 dpsutton didn't mean to derail your exercise. but you can peruse the good if you like 🙂 #2020-06-30 15:58:53 unbalanced I'm hear here to learn in all forms! #2020-06-30 15:59:26 unbalanced Has GraalVM gotten to the point where we can write arbitrary CLI stuff in Clojure now? #2020-06-30 15:59:39 unbalanced (out of the loop) #2020-06-30 15:59:48 borkdude yeah, but you can do a lot with babashka as well, without compiling. #2020-06-30 16:00:01 unbalanced I need to sneak that one in at work 😈 #2020-06-30 16:00:40 borkdude @goomba meta-example: https://github.com/borkdude/deps.clj
that is the clojure bash script ported to both babashka and GraalVM, so you can either run it as script or binary #2020-06-30 16:01:20 unbalanced you can even babashka in powershell!? @borkdude your level of masochism is very impressive #2020-06-30 16:01:34 borkdude yeah, Windows supported too #2020-06-30 16:02:02 unbalanced some heroes don't wear capes #2020-06-30 16:02:32 borkdude how do you know I'm not wearing a cape? (haha, just kidding...) #2020-06-30 16:03:01 unbalanced well with covid I'm willing to wager most aren't even wearing pants. So I was just playing the odds #2020-06-30 16:04:40 unbalanced It looks like find-deps , from a quick glance, is dealing with the same limitation I have #2020-06-30 16:05:00 unbalanced which is finding reagent is easy but org.clojure/core.match or org.clojure/core.async is not as easy #2020-06-30 16:05:27 unbalanced I'm not sure how to differentiate in Maven what is Clojure and what is Java, for instance, conceptually #2020-06-30 16:06:02 unbalanced or... maybe it doesn't matter -- as I say it out loud. That's kind of the point of Clojure, isn't it? 😆 #2020-06-30 16:17:29 dpsutton there are some nice todos and vision in that find-deps lib that you could tackle #2020-06-30 16:17:39 dpsutton dealing with rankings and preferences if i recall correclty #2020-06-30 16:47:23 alexmiller If you’re talking about searching/finding deps, we’ve actually done some thinking about this in tandem with add-lib and I’ve even prototyped some of it in the add-lib2 branch of tools.deps but it’s not in a usable state right now #2020-06-30 16:48:42 alexmiller I guess really more in the context of repl dev not updating deps.edn #2020-06-30 16:49:36 alexmiller But could potentially be used for either I guess #2020-06-30 22:44:32 flefik Is there a good commit on the add-lib or add-lib2 branch that can be checked out and played around with?
I'm thinking of adding a filewatcher on deps.edn and just automatically add any new libraries that are added there without having to restart the repl. #2020-06-30 23:29:06 seancorfield I have an example in my dot-clojure repo that I use all the time -- but @alexmiller has cautioned that add-lib will probably break as code moves around #2020-06-30 23:29:28 seancorfield ^ @UAEFFG05B #2020-07-01 07:47:53 flefik thanks! #2020-06-30 22:50:15 avi No #2020-06-30 22:50:31 avi I meant to ask Alex about it here… thanks for the reminder #2020-06-30 22:51:25 avi The comment on that issue from olpaw to the effect that the tar.gz file shouldn’t have been included in the classpath in the first place made sense to me. #2020-06-30 22:51:53 avi So I thought maybe this might be one of those places wherein tools.deps is deviating from commonly assumed Maven behavior/convention #2020-06-30 22:52:14 avi I don’t really know ¯\(ツ)/¯ #2020-06-30 22:58:54 avi Right #2020-06-30 23:01:28 avi I assume must be some use case for including such a file in a pom, although I have no idea what it is. And I assume that Maven, and whatever mechanism Leiningen uses to work with Maven, somehow filters these files out when/where they’re not needed/useful. #2020-06-30 23:23:56 alexmiller That’s a native dep and there is a facility in Maven to support loading those for the jvm #2020-06-30 23:24:51 alexmiller That Linux and amd64 there are actually parsed and used to select the right native dep for the os/arch #2020-06-30 23:25:22 alexmiller I think lein supports at least some of this too, but deps does not #2020-06-30 23:26:56 lread Thanks @alexmiller! #2020-06-30 23:34:08 lread @aviflax I have something working for my project, I’ll head over to #graalvm to followup #2020-06-30 23:34:22 avi Great! #2020-07-01 11:38:56 practicalli Does the clojure or clj command from Clojure CLI tools read a user.clj file if it contains (ns user) expression when starting the REPL?
With Leiningen projets using a user or dev/user.clj file was one way to start component services.
For a Clojure CLI project, if the dev directory was added as an extra path under a :dev alias and clj -A:dev used to start the REPL, should the dev/user.clj file be automatically evaluate on REPL start?
I though I had this working the other week, but not sure if I imagined this now 🙂 #2020-07-01 11:54:58 practicalli Ah, I have it working in terms of requiring a namespace and evaluating a funtion call. However, using (in-ns main-project-namespace) does not seem to switch to the namespace (or if it does, then maybe it been switched back to user afterward). #2020-07-01 11:57:44 borkdude @jr0cket load-file also switches back to the original namespace afterwards, because it uses binding + *ns* #2020-07-01 11:57:59 borkdude I imagine something similar is happening here #2020-07-01 12:13:36 practicalli Okay, I dont have to change the namespace, I can require the relevant libraries and use fully qualified namespace function calls.
https://practicalli.github.io/clojure/clojure-tools/configure-repl-startup.html
If there is a way in Clojure CLI tools to change the namespace using a user.clj file, I would be interested to know, but not essential. I can use the --eval option to run the relevant expressions. #2020-07-01 12:15:54 borkdude @jr0cket This seems to work:
$ clj -e "(ns foo.bar) (alter-var-root #'*ns* (constantly 'foo.bar))" -r
#2020-07-01 12:17:02 borkdude @jr0cket or even:
$ clj -e "(ns foo.bar) (alter-var-root #'*ns* (constantly 'foo.bar)) (clojure.main/repl)"
foo.bar
foo.bar=>
#2020-07-01 12:18:41 practicalli I can set the namespace easily using --eval option on the command line. Unfortunately the above do not work from within user.clj. Ideally I'd like use it with rebel readline, which is the REPL UI I use if I am just using the REPL directly rather than an editor. --eval doesnt work when using the :rebel alias #2020-07-01 12:22:31 borkdude @jr0cket Very well.
$ clj -R:rebel -e "(ns foo.bar) (alter-var-root #'*ns* (constantly (find-ns 'foo.bar)))" -m rebel-readline.main
#object[clojure.lang.Namespace 0x46cf05f7 "foo.bar"]
[Rebel readline] Type :repl/help for online help info
foo.bar=>
#2020-07-01 12:23:57 practicalli Success 🙂 I have learnt many things today, thank you. #2020-07-01 19:52:49 drewverlee Is there a way to do a universal exlucions? i see you can attach exluded to individual deps, but what if i wont want this lib from any dep? #2020-07-01 19:55:40 borkdude @drewverlee you can use :classpath-overrides:
{:aliases {:babashka {:classpath-overrides {org.clojure/clojure nil
org.clojure/spec.alpha nil
org.clojure/core.specs.alpha nil}}}} #2020-07-01 19:57:37 borkdude This only works in an alias #2020-07-01 19:58:15 drewverlee gotcha. #2020-07-02 13:22:57 arohner is there a library that provides lein deps :pedantic :abort? i.e. looks for and warns on conflicting versions? #2020-07-02 13:23:34 arohner or documentation on how conflicting versions are handled? #2020-07-02 13:27:23 delaguardo you could use -Strace to create edn file with all that information. but I’m not aware of any tooling to make reading of that file easy #2020-07-02 13:28:31 alexmiller there are a few tools like depot #2020-07-02 13:30:13 arohner I’m looking to handle a jarhell situation where I declare dependencies on A and B, and they each declare a dependency on C, but with conflicting versions #2020-07-02 13:31:09 sveri You could refernce the conflicting dependency as a top level dependency and decide for a version yourself #2020-07-02 13:32:12 arohner Right, but I need to discover conflicts before resolving them. I’d like a tool to detect them and warn before I find out the hard way #2020-07-02 13:30:44 arohner :pedantic :abort warns about that, in (some) situations #2020-07-02 13:31:34 arohner but I’d also need to understand how tools.deps resolves that tree. lein’s dependencies are a vector, so deps are ordered sequentially. With a map, how does that work? #2020-07-02 13:33:31 alexmiller well I did whole conj talk about it. it's complicated #2020-07-02 13:34:13 arohner Is the talk uploaded? #2020-07-02 13:33:41 alexmiller but generally the idea is to pick the newest version #2020-07-02 13:34:25 alexmiller but it also picks consistent subtrees (so no child lib should be included if it's parent isn't) #2020-07-02 13:35:27 alexmiller clj -Strace is the best info about what it did and why - the log details each lib/version it looked at (breadth first), what it decided, and why #2020-07-02 13:36:05 alexmiller tools.deps.graph has a tool for using trace.edn files to generate a series of pictures from that that can be somewhat helpful as well #2020-07-02 16:38:48 dominicm My argument is that pedantic abort doesn't make sense in a deps world. #2020-07-02 16:40:59 dominicm You'll never get an older version than what is required, which eliminates 90% of problems you get with maven. In maven, it's assumed everything is conflicting. Deps assumes that newer things are compatible with older things (so the conflict is picking the older dependency?). #2020-07-02 16:42:31 dominicm However, the equivalent tool I think would be useful is detecting a top level dependency on an older artefact, when you've bumped the version and it transitively depends on something newer. #2020-07-02 19:41:26 markbastian If I have a dependency specified in my main deps section and another version of that dependency under an alias, does the alias take precedence when it is active? #2020-07-02 20:15:26 alexmiller yes #2020-07-02 21:04:21 markbastian Is there a way to specify a subdirectory when providing git coordinates for a dependency? For example, the git project is foo and within that you have the folders foo-clj and foo-py. deps.edn is in foo-clj. I want to point deps to the foo-clj folder. #2020-07-02 21:06:03 borkdude @markbastian yes, :deps/root #2020-07-03 15:40:23 arohner is there a way to exclude a dep from the tree, regardless of how it got there? #2020-07-03 15:40:40 arohner :exclusions appears to be per-dep only, but I want to assert e.g. logback should never be present #2020-07-03 19:31:45 neumann Is there a way to replace or remove paths in an alias? I want to exclude on of my source paths in just one alias. #2020-07-03 19:32:52 neumann My current coping mechanism is to set -Sdeps '{:paths ["some" "paths" ...]}' on the command line. #2020-07-03 19:45:36 dominicm @neumann why not move the optional paths to an alias? #2020-07-03 19:50:02 neumann @dominicm I'd like to avoid that because I don't want to have to specify another -A: in all of the cases except one. The extra -A: is just another thing to forget to do. #2020-07-03 19:54:42 borkdude @neumann major hack, this removes the src path from the classpath:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "274548554c4352434267515754161e161f"}, :content ("[email protected]")} #2020-07-03 19:56:24 neumann @borkdude Nice! Yes, I was thinking of maybe using babashka. (Which is amazing! Thank you so much for creating it!) #2020-07-03 20:01:06 neumann @borkdude Something like this, using "src" as the example:
cat deps.edn | bb -I -e '(->> *input* first :paths (remove #{"src"}) vec)' #2020-07-03 20:02:25 borkdude @neumann if you need only one EDN input value:
$ cat deps.edn | bb -e '(->> *input* :paths (remove #{"src"}) vec)'
["parser" "resources" "inlined"] #2020-07-03 20:02:44 borkdude -I is for collecting a lazy seq of multiple EDN input values #2020-07-03 20:03:21 neumann Oh! Very nice! Thanks! #2020-07-04 18:28:36 p-himik If A depends on B, and my project depends on A and B but doesn't really care about the version of B as long as it's compatible with A, how should I specify this dependency on B in deps.edn?
Or should I just omit the dependency on B from deps.edn? #2020-07-04 19:10:41 seancorfield I would figure out what version of B your chosen version of A depends on and explicitly specify that in deps.edn -- and add a comment about the version selection reason. Assuming you mean that your project has an explicit dependency on B. #2020-07-04 19:23:50 p-himik That was exactly my thought initially. But now that I've given it some thought - wouldn't it be more practical to just omit it altogether?
If A starts to depend on a different version of B, I should be unaffected, and I won't need to change anything.
If A drops the dependency, the code will just not run because there's an explicit :require.
And specifying B in my deps.edn seems like just giving myself more work when I update my dependencies. #2020-07-04 19:33:03 seancorfield If A changes the version of B it depends, it also changes its own version -- so unless you are changing the version of A you depend on, you wouldn't be affected 🙂 #2020-07-04 19:34:04 seancorfield That's my point: if you choose to depend on a newer version of A, you should also verify the compatibility of the newer version of B that it now depends on as well. Having an explicit dependency on B lets you force the earlier version of B if you need that. #2020-07-04 19:35:18 seancorfield If A stops depending on B, that also means a new version of A and you could decide to stay with the old version or move to the new version -- and because you have an explicit dependency on B, you code won't break (at least, not because A's transitive dependency on B went away). #2020-07-04 19:36:23 seancorfield I really don't like relying on transitive dependencies. I much prefer to have all my dependencies listed explicitly. I've learned that over the years after a decade of Clojure in production 🙂 #2020-07-04 19:43:35 p-himik Hmm, I think I didn't give enough context.
I definitely agree with your reasoning about the generic case of relying on a transitive dependency. But in my particular case, the code should either break completely if B is changed/removed, or it should continue working just as I expected it to. So the compatibility with my project is not a concern.
At the same time, if I manage to forget to update B after updating A, I may introduce some incompatibilities between A and B that are hard to test. #2020-07-04 19:44:57 p-himik To be more concrete - I'm just using ring.util.mime-type/ext-mime-type function without having an explicit dependency on ring/ring-core. #2020-07-04 19:46:36 p-himik Of course even such a dead simple function can potentially be broken with an update. But I consider it so improbable that it's not worth taking into account. Akin to how I don't expect clojure.core/map to suddenly start giving me different results. #2020-07-04 19:53:59 seancorfield But you have code in your project that depends on Ring namespaces? #2020-07-04 19:54:42 seancorfield To be concrete, what are A and B specifically in your project? #2020-07-04 20:02:09 p-himik A - yada/core
B - ring/ring-core
I need only ring.util.mime-type/ext-mime-type from B, which I consider stable in a sense that any changes around it either break the app startup or don't affect the app at all. #2020-07-04 20:02:30 p-himik I don't depend on anything ring-related but that small function. #2020-07-04 20:08:20 seancorfield Ah, OK. If that is the only Ring namespace you're touching then maybe I would agree and just depend on Yada. It still feels a bit sketchy to me. We depend on Ring and we don't use that var -- we have our own MIME type structures (because we want a specific set of supported values, rather than a generic list). #2020-07-04 20:13:21 p-himik I wish I would have a specific set. :)
Yeah, I can see how it can feel sketchy. I'm still not sure about the way I work with my dependencies at all right now, given that there's also CLJS and a build process that involves shadow-cljs. Maybe this all could be made nice and simple with aliases. An unrelated to the rest of the thread question - aliases cannot be grouped together under a new alias, right? So I will probably have to write a script to call clj -A:al1:al2:al3. #2020-07-04 20:20:04 seancorfield Nope. Grouping of aliases is a common request. #2020-07-04 20:21:06 p-himik I see.
Anyway, thanks for the input Sean. Much appreciated. :) #2020-07-04 20:22:09 seancorfield It's why at work we have a build shell script that wraps the clojure script so we can have "higher-level" aliases. #2020-07-04 20:24:05 p-himik Right, I've seen this pattern before. Sometimes even with -Sdeps, to avoid cluttering the main deps.edn I guess or to make the script more portable. #2020-07-04 20:38:52 seancorfield Our build script serves two purposes: to allow us to run a series of clojure commands from a single invocation, and to provide various shorthands for common combinations of aliases (our aliases tend to be very granular). But we're all Clojure / backend. I suspect if we were managing ClojureScript as well, we'd be even more reliant on our wrapper script! #2020-07-04 20:41:14 seancorfield I'm looking forward to tools.build -- I suspect it will help simplify our tool chain quite a bit. #2020-07-04 20:49:21 p-himik I don't think I've heard of it and I can't find anything that seems relevant. Is there an announcement or something like that? #2020-07-04 20:59:35 seancorfield Alex mentioned it in an Inside Clojure blog post and has mentioned it several times here. #2020-07-04 21:00:53 p-himik Heh, found it: "tools.build - shh, top secret" #2020-07-04 21:06:05 seancorfield Giving Cognitect's solid track record of producing simple, well-considered, composable tooling that addresses core Clojure developer needs, I'm looking forward to whatever this turns out to be 🙂 #2020-07-05 08:36:05 dominicm @p-himik I've been considering writing a tool which ensures your top level dependencies are at least newer than those required by your dependencies #2020-07-05 08:38:27 p-himik That may be useful. :+1: Although not for my problem in particular because I ended up just dropping the explicit dependency. :) #2020-07-06 10:36:09 dominicm I think that's wrong. You depend on that library. #2020-07-06 10:50:22 p-himik We've discussed that at length with Sean just a bit above. :) It's not that wrong in my particular context, I think. #2020-07-06 10:52:04 p-himik tl;dr: I depend on the library, but I don't depend on the version of the library. And if something breaks, the app just won't start. So by not using an explicit dependency, I cannot really run time runtime errors.
But with an explicit dependency, I definitely can if I forget to update it along with other libraries that depend on it as well. #2020-07-05 13:06:48 alexmiller it would probably be pretty easy to read a trace file (or use the api and the trace data returned) to see that #2020-07-06 10:36:57 dominicm Oh really? Interesting. I'd been resolving each dependency independently and comparing to the top level. Your idea is better. #2020-07-06 10:37:35 dominicm I wonder if you could build a whole lint suite out of the trace, catching other common errors (hello Jackson and jetty). #2020-07-06 11:40:34 fabrao Hello all, how to listen in all interfaces for this
:aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:accept,clojure.core.server/repl} -Dfile.encoding=UTF-8"]}}
? #2020-07-06 11:42:31 fabrao I tried :address,10.10.20.whatever and it didn´t work #2020-07-06 11:45:59 fabrao -> Exception in thread "main" java.lang.NumberFormatException: Invalid number: 10.10.20.82 #2020-07-06 12:33:13 borkdude @fabrao try escaped double quotes around the IP #2020-07-06 13:09:23 fabrao @borkdude :aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"10.10.20.82\",:accept,clojure.core.server/repl} -Dfile.encoding=UTF-8"]}} -> same error #2020-07-06 13:09:58 fabrao I tried with ' too, not working #2020-07-06 14:02:37 alexmiller that seems weird to me. can you share the full output? #2020-07-06 14:16:07 fabrao Exception in thread "main" java.lang.NumberFormatException: Invalid number: 10.10.20.82
at clojure.lang.EdnReader.readNumber(EdnReader.java:224)
at clojure.lang.EdnReader.read(EdnReader.java:136)
at clojure.lang.EdnReader.readDelimitedList(EdnReader.java:766)
at clojure.lang.EdnReader$MapReader.invoke(EdnReader.java:680)
at clojure.lang.EdnReader.read(EdnReader.java:145)
at clojure.lang.EdnReader.read(EdnReader.java:111)
at clojure.lang.EdnReader.readString(EdnReader.java:67)
at clojure.edn$read_string.invokeStatic(edn.clj:46)
at clojure.edn$read_string.invokeStatic(edn.clj:37)
at clojure.core.server$parse_props$fn__8907.invoke(server.clj:152)
at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
at clojure.core.protocols$fn__8159.invokeStatic(protocols.clj:169)
at clojure.core.protocols$fn__8159.invoke(protocols.clj:124)
at clojure.core.protocols$fn__8114$G__8109__8123.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__8136.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__8136.invoke(protocols.clj:75)
at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6828)
at clojure.core.server$parse_props.invokeStatic(server.clj:146)
at clojure.core.server$start_servers.invokeStatic(server.clj:157)
at clojure.core.server$start_servers.invoke(server.clj:157)
at clojure.lang.Var.invoke(Var.java:384)
at clojure.lang.RT.doInit(RT.java:493)
at clojure.lang.RT.init(RT.java:467)
at clojure.main.main(main.java:38) #2020-07-06 14:20:02 alexmiller you need to put that second -D in a separate string #2020-07-06 14:21:44 alexmiller {:aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"10.10.20.82\",:accept,clojure.core.server/repl}" "-Dfile.encoding=UTF-8"]}}} #2020-07-06 14:21:55 fabrao let me try #2020-07-06 14:22:02 alexmiller I can't repro what you're getting #2020-07-06 14:22:14 alexmiller I get an error binding that address, but that's to be expected #2020-07-06 14:26:37 alexmiller I would expect that error w/o the quotes around the address #2020-07-06 14:27:28 borkdude @fabrao Are you doing this from a deps.edn file or from a cmd line with -Sdeps? #2020-07-06 14:31:14 fabrao deps.edn #2020-07-06 14:34:01 borkdude linux, macOS, Windows? #2020-07-06 14:39:50 markbastian From what I've read, it sounds like a fairly standard approach to publishing library artifacts with deps is to create a pom with clj -S:pom and then use mvn deploy. When you do this model, are there any best practices regarding how to keep things in sync? For example, do you:
1. Regenerate the pom every time you want to cut a release? This seems wrong as you lose all of your version and other info.
2. Generate the pom once and then maintain it? This seems like a good route, but the main downside I see is keeping dependencies in sync.
I am assuming #2 is the best option. Any tips for syncing dependencies? #2020-07-06 15:35:16 seancorfield @markbastian clojure -Spom does not overwrite anything in your pom.xml except <dependencies> #2020-07-06 15:48:07 alexmiller And repositories and source dir #2020-07-06 15:35:45 seancorfield And for publishing to Clojars, you really need a lot more in your pom.xml file than clojure -Spom provides. #2020-07-06 15:38:21 seancorfield If you create a new project with clj-new (typically clj -A:new lib myname/myproject) it will have a suitable pom.xml file and then for each release you just run clojure -Spom, update the version and tag in the pom for the upcoming release, commit and push those pom changes, add a release on GitHub and pull, clojure -A:jar (because clj-new makes that alias available), and then clojure -A:deploy (again, from clj-new's template) #2020-07-06 15:39:32 markbastian Ah, I hadn't realized what the pom command did. I thought it generated a pom (replacing the existing). I just did an experiment based on your comment and suddenly things make a lot more sense. Very cool! Thanks for the info. #2020-07-06 15:42:21 seancorfield Per clojure -h:
-Spom Generate (or update an existing) pom.xml with deps and paths
#2020-07-06 15:40:03 seancorfield The :jar alias is for depstar and it gets the group, artifact, and version from pom.xml and builds everything you need inside the .jar incl. manifest, multi-release, etc. #2020-07-06 15:40:44 fabrao @borkdude It´s Windows #2020-07-06 15:40:55 fabrao :aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"10.10.20.82\",:accept,clojure.core.server/repl}" "-Dfile.encoding=UTF-8"]}} didn´t work #2020-07-06 15:40:56 borkdude aaah #2020-07-06 15:41:00 seancorfield The :deploy alias is for deps-deploy and it expects
CLOJARS_PASSWORD=<your secret>
CLOJARS_USERNAME=<your username>
to be set in your env vars #2020-07-06 15:41:30 borkdude @fabrao windows is a bit different here maybe. can you try "" instead of \"? #2020-07-06 15:41:53 borkdude oh no, that makes the EDN file invalid #2020-07-06 15:41:54 borkdude hmm #2020-07-06 15:42:02 borkdude try two \"'s #2020-07-06 15:43:35 borkdude or just try WSL2 🙂 #2020-07-06 15:44:07 borkdude but I suspect doubling the double quotes works #2020-07-06 15:44:55 fabrao \"\"10.10.20.82\"\" and ""10.10.20.82"" -> Invalid Number #2020-07-06 15:45:38 borkdude @seancorfield he isn't using -Sdeps from the command line, this is in an EDN file #2020-07-06 15:45:57 seancorfield I am pretty sure the same recommendations apply... #2020-07-06 15:46:44 fabrao sorry ->
Error: Could not find or load main class 10.10.20.82
#2020-07-06 15:46:57 fabrao Caused by: java.lang.ClassNotFoundException: 10.10.20.82
#2020-07-06 15:47:25 seancorfield @fabrao There's a #clj-on-windows channel that has real Windows users that have probably solved this problem. You're just kind of flailing right now... #2020-07-06 15:48:28 fabrao ok Sean #2020-07-06 15:48:37 seancorfield (I would probably try \\" in the string in deps.edn tho') #2020-07-06 15:51:02 fabrao with \\" didn´t work too #2020-07-06 15:52:02 seancorfield I'm firing up a Windows VM. I'll try a few years and report back in #clj-on-windows #2020-07-06 16:01:40 borkdude @fabrao, this worked for me on Windows:
{:aliases {:dev {:jvm-opts ["-Dclojure.server.repl={:port,5555,:address,\"\"\"127.0.0.1\"\"\",:accept,clojure.core.server/repl}" "-Dfile.encoding=UTF-8"]}}}
so doubling the escaped quotes, just like I suspected 😎 #2020-07-06 20:08:38 seancorfield Technically that is tripling the escaped quotes #2020-07-06 20:09:14 borkdude oh wait, I didn't even know I did that. #2020-07-06 20:09:51 seancorfield And weird that it seems to work across all platforms... 😐 #2020-07-06 20:11:45 borkdude haha #2020-07-06 16:01:59 borkdude and I verified with netstat that the server was indeed running with clojure -A:dev #2020-07-06 16:19:33 seancorfield (sorry, got distracted by a phone call with my mum in England, but I was getting there I hope...) #2020-07-06 16:39:39 seancorfield FWIW, that format with \"\"\" around the host IP seems to work on Linux and macOS as well. Which surprised me a bit. I was able to start a REPL with that alias on Powershell, CMD (via powershell clj), and WSL -- as well as on macOS (Terminal) -- and connect to it via telnet to verify the socket REPL connection. #2020-07-06 18:18:10 cap10morgan Is there a way to just install the deps w/ tools-deps? I'm looking for an equivalent to lein deps (for caching them in a Docker image build). #2020-07-06 18:21:08 borkdude @cap10morgan you can just do -Spath and will print the classpath, downloading the deps as a side effect #2020-07-06 18:27:47 seancorfield -Spath, -Stree, and -e nil seem to be the most commonly mentioned ways to have the CLI just download deps, optionally print some information about them, and just exit. #2020-07-06 18:28:15 seancorfield I like -Stree myself since it's easier to read than -Spath if I want to verify the actual dependencies being used. #2020-07-06 18:28:24 seancorfield Or just -e nil if I don't care. #2020-07-06 18:29:53 cap10morgan Great. Thanks! #2020-07-06 19:36:23 fabrao @borkdude It worked !!! Strange things #2020-07-06 19:47:21 borkdude @fabrao Might be good to add to the wiki. #2020-07-06 23:47:53 Spencer Apple With the monorepo root CLJ_CONFIG deps.edn
{:aliases {:dev {:jvm-opts ["-XX:-OmitStackTraceInFastThrow"]}}}
And a local deps.edn
{:aliases {:dev {:extra-deps {...}}
I can pull configuration from the monorepo root CLJ_CONFIG
CLJ_CONFIG=../.clojure clojure -A:dev
However, this doesn’t pass down the :jvm-opts from the CLJ_CONFIG deps.edn. Is this working as intended? Is there a better way to do this? (I could create another alias with this jvm-opt or perhaps use environment variables.) #2020-07-06 23:48:20 seancorfield Aliases do not compose. #2020-07-06 23:49:26 Spencer Apple there’s a line in the guide:
> If multiple maps with these keys are activated, `:jvm-opts` concatenate #2020-07-06 23:49:32 Spencer Apple that confused me #2020-07-06 23:50:42 seancorfield Right, that means across all the aliases you specified. #2020-07-06 23:50:52 Spencer Apple ah gotcha, thanks for the help! #2020-07-06 23:51:08 seancorfield But they have to be different aliases. I answered in the main channel how we handle this at work. #2020-07-06 23:50:18 seancorfield You can either have an alias in the user-level file (via CLJ_CONFIG) or in the project-level file. What we did is to have something like :dev-defaults in the user-level (monorepo) file and then :dev as an empty alias there and overridden in each project -- and then do CLJ_CONFIG=../defaults clojure -A:dev-defaults:dev #2020-07-06 23:51:56 Spencer Apple great, this seems like a good solution!
And thanks for the quick answer 😄 #2020-07-06 23:52:40 seancorfield We have about thirty subprojects in our monorepo (and over 100,000 lines of code overall). #2020-07-06 23:53:41 Spencer Apple Thank you for sharing your deps.edn btw, I should have studied that a bit more closely… #2020-07-06 23:55:50 seancorfield LMK if you have any Qs -- feel free to DM me any time about it. #2020-07-07 15:44:43 pmonks @seancorfield might be time to consider creating a #seans-deps-edn channel here? 😉 #2020-07-07 23:13:40 pmonks Is there a preference or recommendation between these two (equivalent) styles of specifying a version:
{:deps {some/library {:mvn/version "2.7.42"}}}
and
{:deps {some/library #:mvn {:version "2.7.42"}}}
?
(yes I realise this is a stylistic question not a technical one - I’m wondering if there’s a preferred / recommended / idiomatic style for deps.edn content) #2020-07-07 23:32:16 hiredman I'd say namespaced maps are not mentioned in https://github.com/edn-format/edn, so if deps.edn is intended to live up to the .edn on the end, leave out the namespaced maps #2020-07-07 23:34:47 pmonks Ooooh that’s very interesting! Thanks! #2020-07-08 00:30:27 alexmiller the Clojure edn reader does read namespace maps but hiredman is correct that this is not currently part of the spec #2020-07-08 00:07:53 alexmiller the first #2020-07-08 09:07:30 holyjak Hello! How do I run clj to only download dependencies for a profile and exit at once? Like this: clojure -R:nrepl -e "(println :DOWNLOADED)" ? Thanks! #2020-07-08 09:39:20 dominicm @holyjak clojure -R:nrepl -Spath should do it :) #2020-07-08 11:00:23 holyjak thanks!!! #2020-07-08 19:22:16 fabrao hello all, what is the magic about starting so fast in comparing with lein and boot ? #2020-07-08 19:25:28 souenzzo - less deps, clean design
- do less things (lein/boot do A LOT more stuff)
- cache classpath "in bash"
... #2020-07-08 19:25:55 borkdude fire up only one java process (when classpath is cached) #2020-07-08 19:27:50 souenzzo design choise example:
lein help will build a classpath, start a JVM, load all plugins, ask to every plugin if it implements "help" and eventually show the help message
clj --help is handled without start any JVM #2020-07-08 19:51:15 fabrao this saved me time and memory, many thanks who made it #2020-07-09 11:16:33 ejelome ...
Hello, I have a deps.edn question, I tried to replicate what I use in the terminal as config options with deps.edn, but it seems that the generated .js file logs some errors,
Here's the CLI:
clj -m cljs.main -o public/js/main.js -O advanced -c hello-world.core
Here's what I expected as equivalent deps.edn version:
{:deps {org.clojure/clojurescript {:mvn/version "1.10.764"}
reagent {:mvn/version "1.0.0-alpha2"}}
:main "hello-world.core"
:optimizations :advanced
:output-dir "public/js/out"
:output-to "public/js/main.js"}
Then I run it with:
clj -m cljs.main -co deps.edn
The generated .js file from CLI works fine, but the generated version from deps.edn options gives an error:
main.js:314 Uncaught TypeError: Cannot read property 'mg' of undefined
at Mb (main.js:314)
at Tn (main.js:699)
at Rn (main.js:698)
at Nn.h.Me (main.js:718)
at Function.Wn.A (main.js:710)
at Nn.h.Le (main.js:718)
at Function.Wn.j (main.js:710)
at ep (main.js:761)
at main.js:761
... and it doesn't seem to exit from the terminal (the terminal becomes unusable) #2020-07-09 11:26:34 euccastro you may want to try asking in #clojurescript? #2020-07-09 11:27:29 ejelome hey, thanks, that's probably the right place #2020-07-09 19:13:37 psetmaj Are there any default aliases or good ways to get some extra resource paths added to the classpath for all local repls started with the clojure command? (some key in ~/.clojure/deps.edn perhaps?)
I’d prefer not to need to add a -A flag to all invocations across the several IDEs I use #2020-07-09 19:14:31 alexmiller if you mean an alias that is automatically supplied, no #2020-07-09 19:14:57 alexmiller :paths in ~/.clojure/deps.edn will be used everywhere #2020-07-09 19:15:23 psetmaj I thought :paths got merged across the candidate deps.edns with a “last one wins” strategy #2020-07-09 19:16:42 alexmiller oh yes, sorry #2020-07-09 19:21:09 psetmaj Does :extra-paths only exist for aliases? My use case is that I’ve got a local directory for various properties files that’s global to all projects (shared DB/service locations, and whatnot for local development, shared across several languages). #2020-07-09 19:33:28 alexmiller yes, only for aliases #2020-07-09 19:48:15 psetmaj Okay, thanks for your responsiveness and time 🙂 #2020-07-09 22:12:57 souenzzo @alexmiller I see that :git/url "file:///foo" isn't supported by gitlibs
I already find that it's a issue with clojure.tools.gitlibs.impl/clean-url
A PR/Patch is welcome about this problem? #2020-07-09 22:13:36 alexmiller I'm unclear why this is better than using :local/root #2020-07-09 22:14:26 alexmiller git deps let you get the dep from a remote git repo. if you already have it locally, just use it? #2020-07-09 22:14:42 alexmiller I guess a git dep would let you use multiple shas #2020-07-09 22:14:49 alexmiller can you walk back and say how you're using it? #2020-07-09 22:15:13 alexmiller stepping away for evening but will check later #2020-07-09 22:19:01 souenzzo Let's go:
I'm trying to deploy my application with clojure -Sdeps "{:deps {my/app {:git/url \"...\" :sha \"...\"}}}" -m my.app.start
But I'm on AWS/CodeCommit
CodeCommit on ephemeral EC2 recommends use https with credential provider
jgit do not support credential provider
so I'm trying to use git-cli to clone, then use local file
I can run directly but I want to keep my SHA control in the "start command line" #2020-07-09 23:29:57 souenzzo TL; DR: How to use git deps with CodeCommit? #2020-07-10 11:27:42 jeroenvandijk @U2J4FRT2T You have to put some magic in your .gitconfig . Something like:
[credential ""]
helper = !aws --profile dev1 codecommit credential-helper
And maybe enfore https (not sure, don’t remember):
[url ""]
insteadOf =
#2020-07-10 11:28:49 jeroenvandijk If I remember correctly this did work with tools.deps #2020-07-10 16:13:09 kenny Does anyone know of an up-to-date guide on deploying a jar to a private maven s3 repo? I've tried several different guides and they all reference deprecated or archived maven build plugins. It seems like such a simple task but there seems to be dozens of different ways to get it done. #2020-07-10 16:31:07 kenny Oh interesting... it appears all I need to do is
aws s3api put-object --bucket my-bucket --key release/my-group-id/my-artifact-id/my-version/my.jar
#2020-07-10 18:06:15 kenny Actually, tools-deps does not seem completely satisfied with the above approach, printing Download corrupted: Checksum validation failed, no checksums available #2020-07-10 18:06:40 kenny Guessing it expects some additional files in s3 that are not placed via the simple put-object action. #2020-07-10 19:18:38 drewverlee If i have a lib calling a method that doesn't exist and i suspect its because two version of the lib exist and the one without it won, how can i verify this? #2020-07-10 19:19:22 drewverlee the stacktrace lists the class package and class name and i compared this against the output of Stree and their isn't a direct overlap #2020-07-10 19:20:08 drewverlee likley because stree is output a higher level of information #2020-07-10 19:24:52 seancorfield @drewverlee Which lib? Which class? Which method? #2020-07-10 19:27:13 drewverlee method: ""com.google.common.collect.ImmutableList.toImmutableList()Ljava/util/stream/Collector;"
lib that calls it: pack/pack.alpha https://github.com/juxt/pack.alpha.git @ 0c20aa5 #2020-07-10 19:28:12 drewverlee well more accurately:
[{:type java.lang.NoSuchMethodError,
:message
"com.google.common.collect.ImmutableList.toImmutableList()Ljava/util/stream/Collector;",
:at
[com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep
makeList
"BuildAndCacheApplicationLayerStep.java"
67]}]
#2020-07-10 19:29:50 drewverlee so google.cloud.tools.jib.builder calls com.google.common.collect.ImmutableList.toImmutableList and it doesn't exist. I assume that error isn't saying the arguments were bad in someway, its saying that method isn't there at all. #2020-07-10 19:38:08 seancorfield And what Google libs show up in clojure -Stree for you? #2020-07-10 19:38:37 seancorfield You may need to run clojure -Strace and see what other versions of the various Google libs were considered. #2020-07-10 19:40:37 drewverlee > And what Google libs show up in clojure -Stree for you?
I see guava listed under one dep i just added to my project. Is that because Stree only shows the winner of the dep merge? I suppose i'm looking for everyone that tried to add it.
I'll look at Strace more carefully. #2020-07-10 19:45:21 drewverlee i see Strace contains anyone that wanted to add the dep. isee it all over the place here. #2020-07-10 19:50:10 seancorfield The output of -Strace should show you every version of every library that it considered and why it made the decision it did to pick a particular version. #2020-07-10 19:52:49 seancorfield So you could just add a top-level dep on Guava to force a recent version, but I'm surprised tools.deps would pick a lower version, since I would expect the Google Cloud libs you're using to pull in at least that version or higher... #2020-07-10 19:54:04 seancorfield According to Guava's docs, it's up to 28.0 now. 👀 #2020-07-10 19:54:55 seancorfield Hmm, odd, we're using 29.0 at work... #2020-07-10 19:55:56 seancorfield 21.0 is about 3 1/2 years old at this point. 29.0 came out in April this year @drewverlee #2020-07-10 19:56:59 seancorfield We've had to pin it as a top-level dependency because other things were bringing in older versions. Java's equivalent of "DLL hell" 🙂 #2020-07-11 01:05:56 tolitius what is a tools.depsy way of starting (--interactive) REPL-y connected to nrepl?
:repl {:extra-deps {nrepl {:mvn/version "0.7.0"}
reply {:mvn/version "0.4.4"}}
:main-opts ["-e" "(require,'[reply.main]),(defn,reply,[_,port,_],(reply.main/launch-nrepl,{:attach,(str,\"127.0.0.1:\",port)})),(ns,user)"
"-m" "nrepl.cmdline" "-i" "-f" "user/reply"]}}
this works (ish), but feels complex + does not exit nrepl once REPL-y is done (ctrl c / ctrl d) #2020-07-11 10:57:47 euccastro do you want to start your nREPL server with the same command? #2020-07-11 10:57:55 euccastro I have this:
{:extra-deps {reply {:mvn/version "0.4.4"}}
:main-opts ["-m" "reply.main" "--attach" "1337"]} #2020-07-11 10:58:13 euccastro but that's to attach to an already running nREPL #2020-07-11 14:39:26 tolitius right nrepl + REPL-y connected to it. I do most of the work in a vi + terminal REPL #2020-07-11 14:39:55 tolitius *right => "start your nREPL server with the same command" #2020-07-11 14:41:06 tolitius nrepl.cmdline optionally takes a repl-fn / -f for the interactive repling #2020-07-11 14:56:41 seancorfield I think, for simplicity's sake, I'd be inclined to use two separate aliases: one to start an nREPL server (and leave it running in a window, or run it in the background), and a second alias to start REPL-y connected to that background process. #2020-07-11 14:58:04 seancorfield I can understand wanting a single command, but I'd probably write a small shell script that just ran two clojure commands if it proved too annoying to run those two commands manually. #2020-07-11 17:18:08 tolitius @seancorfield definitely a possibility, but if I work with several (different) apps that's a twofold repl things to run and exit from
I am just looking for "lein repl" / "boot repl" repl-y like experience with deps.edn #2020-07-11 17:21:38 seancorfield Hmm, I never type into a REPL so I only care what my editor integration does. Why do you need REPL-y, specifically @tolitius? #2020-07-11 17:21:57 tolitius I always type into repl ) #2020-07-11 17:22:52 tolitius it's connected to vi where I "finalize" my thoughts after I am done thinking in repl #2020-07-11 17:23:20 tolitius and vice versa compile/reload from vi back to repl tp keep thinking #2020-07-11 17:23:32 seancorfield I "think" in a "Rich Comment Form" in my editor -- like Stu Halloway talks about in some of his presos #2020-07-11 17:30:11 tolitius yea, a lot of Clojure devs use an integrated playground to "think" that is just connected to nrepl that runs in the background. I like terminals, terminal repl works best for me since I am focused on small fragments of code at a time.
with REPL-y this experience is simple, so I am just looking to get there with deps.edn #2020-07-11 17:30:55 tolitius which I did get with an above alias but it rather feels off + does not exit the nrepl on ctrl c/d #2020-07-11 17:31:05 seancorfield I think you're stuck with the complex/almost-works invocation for now. I'll have a think about it on and off and come back if I figure out anything better. #2020-07-11 17:31:40 tolitius yea, nothing urgent, I was just wondering whether I am missing something obvious, i.e. a flag, an option, etc.. #2020-07-11 17:31:45 seancorfield (I don't use nREPL at all so it'll be a fun experiment/diversion) #2020-07-11 17:31:57 tolitius good luck and thank you ) #2020-07-11 19:28:22 richiardiandrea @tolitius Hey 😉
I am using https://github.com/bhauman/rebel-readline
For your exact use case and I found some problems with multi-line expressions sometimes, I am pretty happy with it.
I have two aliases I can share (the second is for cljs) not in front of the keyboard now though. Will sent them soon. #2020-07-12 16:33:03 tolitius @richiardiandrea niice:
:repl {:extra-deps {nrepl {:mvn/version "0.7.0"}
cider/cider-nrepl {:mvn/version "0.22.4"}
com.bhauman/rebel-readline {:mvn/version "0.1.4"}}
:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]" "-i" "-f" "rebel-readline.main/-main"]}
this works: connects to an editor, reloads recompilations in repl from editor, and also I can now even see some color as a positive side effect )
any differences you found between rebel-readline and REPL-y (that REPL-y does better)? #2020-07-12 17:27:55 richiardiandrea Not much, as I mentioned sometimes rebel's multi-line support has glitches but it was never bothering so much to even open an issue (I should).
Nice thanks for sharing I do something similar myself. #2020-07-12 17:34:09 tolitius that's great, I am still mostly on boot due to little things like this in "deps.edn". but I am slowly moving over )
thanks a lot for a "rebel-readline" idea! #2020-07-12 23:58:11 richiardiandrea my pleasure 😄 #2020-07-13 16:55:35 Space Guy When I used rebel-readline with a background process (Figwheel {:mode :serve} , http-kit, etc), the default rebel-readline/-main won't automatically end those processes if you quit with 'ctrl+d'.
Looks like REPL-y does (shutdown-agents) when done, I haven't checked what nrepl.cmdline does with that specific case
It's not a bug, just a difference (https://github.com/bhauman/rebel-readline/issues/141)
In my case I decided to start both then just exit when done:
(figwheel.main.api/start {:mode :serve} "dev")
(rebel-readline.main/-main)
(System/exit 0)
I currently run this in a clj dev.clj, otherwise a namespace with a main method is the same #2020-07-13 21:35:08 Ian Fernandez hello folks, some help on depstar pls (edited)
clj -C:build -A:build -m hf.depstar.uberjar target/abc.jar -C -m abc.core (edited)
this is giving me
Could not locate abc/core__init.class, abc/core.clj or abc/core.cljc on classpath.
but I have a src/abc/core.clj with (:gen-class)
some clue? #2020-07-13 21:48:59 hiredman when you say it is giving you that, do you mean when you get that when building the uberjar or trying to run it? #2020-07-13 21:57:56 seancorfield Since you're specifying -A:build you don't need to also specify -C:build -- -A means "all" so it already includes the effect of the -C option. #2020-07-13 21:58:19 seancorfield What is your :build alias in your deps.edn @d.ian.b? #2020-07-13 21:59:08 Ian Fernandez :build {:extra-deps {seancorfield/depstar {:mvn/version "1.0.94"}}} #2020-07-13 22:00:20 seancorfield And what exact output are you getting from that clj -A:build command? (I'm assuming the Could not locate... error is from trying to run the JAR?) #2020-07-13 22:01:44 Ian Fernandez Compiling abc.core ...
Execution error (FileNotFoundException) at hf.depstar.uberjar/run$fn (uberjar.clj:331).
Could not locate abc/core__init.class, abc/core.clj or abc/core.cljc on classpath.
Full report at:
/tmp/clojure-5041631337400948497.edn
#2020-07-13 22:02:31 Ian Fernandez clj -A:build -m hf.depstar.uberjar target/abc.jar -C -m abc.core #2020-07-13 22:04:51 seancorfield So that says it's compiling norach.core -- which means depstar thinks that is the main class/namespace you told it to compile. #2020-07-13 22:05:40 Ian Fernandez ops #2020-07-13 22:05:41 Ian Fernandez srr #2020-07-13 22:07:17 Ian Fernandez ops #2020-07-13 22:07:31 Ian Fernandez it was abc.core #2020-07-13 22:07:47 Ian Fernandez I've tried to anonimize =( #2020-07-13 22:07:49 Ian Fernandez the names #2020-07-13 22:07:57 Ian Fernandez and did it wrong #2020-07-13 22:08:21 seancorfield It really helps if you paste actual commands and actual error messages so you avoid transcription problems. #2020-07-13 22:09:23 seancorfield I just did this:
$ clj -A:new app norach.core
... creates norach.core project ...
$ cd norach.core
$ clj -Sdeps '{:deps {seancorfield/depstar {:mvn/version "RELEASE"}}}' -m hf.depstar.uberjar target/abc.jar -C -m norach.core
Compiling norach.core ...
Building uber jar: target/abc.jar
Processing pom.xml for {norach/norach.core {:mvn/version "0.1.0-SNAPSHOT"}}
$ java -jar target/abc.jar
Hello, World! #2020-07-13 22:09:33 Ian Fernandez clj -A:build -m hf.depstar.uberjar target/norach.jar -C -m norach.core
Compiling norach.core ...
Execution error (FileNotFoundException) at hf.depstar.uberjar/run$fn (uberjar.clj:331).
Could not locate norach/core__init.class, norach/core.clj or norach/core.cljc on classpath.
Full report at:
/tmp/clojure-5874121970680475144.edn
#2020-07-13 22:10:17 seancorfield So that indicates that your deps.edn file is not setup correctly -- so norach/core.clj is not on your classpath. #2020-07-13 22:10:41 seancorfield Can you run this: clj -m norach.core #2020-07-13 22:12:07 Ian Fernandez Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate norach/core__init.class, norach/core.clj or norach/core.cljc on classpath.
Full report at:
/tmp/clojure-13768805695698659229.edn #2020-07-13 22:12:08 Ian Fernandez =( #2020-07-13 22:12:29 Ian Fernandez nope #2020-07-13 22:12:35 seancorfield That confirms it's not a depstar problem -- it's a problem with your project. #2020-07-13 22:12:41 Ian Fernandez thanks #2020-07-13 22:13:08 seancorfield You're sure you have src/norach/core.clj and that file starts with (ns norach.core ...) ? #2020-07-13 22:13:34 Ian Fernandez dammit, misstypo #2020-07-13 22:13:39 Ian Fernandez ns norahc.core #2020-07-13 22:13:52 Ian Fernandez dammit #2020-07-13 22:13:54 Ian Fernandez thanksss #2020-07-13 22:18:32 Ian Fernandez Thanks @seancorfield 😃 #2020-07-14 11:54:21 eval2020 Any chance that tda will support providing multiple config files? e.g. by allowing CLJ_CONFIG to be ~/.clojure:/considered/first
This would be great to share a deps.edn via dotfiles or in a monorepos.
context: I'd like to curate a deps.edn for the team I'm working with. This file would contain aliases and overrides that are now duplicated across repositories.
This file should exist along a user's ~/.clojure/deps.edn .
E.g. CLJ_CONFIG=~/.clojure:~/.clojure/work clojure -Sdescribe would result in ...:config-files ["<global>/deps.edn" "~/.clojure/deps.edn" "~/.clojure/work/deps.edn" ?"$PWD/deps.edn"] ensuring that within the team we share aliases while allowing for user-specific aliases and versions (e.g. for editor setup).
In a monorepos I can see this being helpful as well. E.g. with the following project setup and a global export CLJ_CONFIG=~/.clojure :
.
├── shared/deps.edn
└── subproject1/deps.edn
doing export CLJ_CONFIG=$CLJ_CONFIG:/path/to/project/shared as part of a project setup (or in scripts) would allow one to move through the project and have a config-files collection of ["<global>/deps.edn" "~/.clojure/deps.edn" "/path/to/project/shared/deps.edn" ?"$PWD/deps.edn"] #2020-07-14 12:15:47 dominicm Write a tool to update the deps edn file in each repo #2020-07-14 12:43:20 eval2020 @U09LZR36F thought about going that way but seems hard to maintain across a lot of repositories... #2020-07-14 13:43:22 dominicm Why's that #2020-07-15 14:05:13 eval2020 In our case it would be the amount of repositories (50-ish). Also the contents of the deps.edn: some contain a linter alias, others a clj-kondo, any docs for existing alias. And then doing it for every update to the shared stuff.
Have you successfully applied this approach? #2020-07-15 15:23:44 dominicm Not to 50 or so repos. But if you want people to actually update, then you probably want to push it into their repo, instead of hoping that they will update it on their local machine. #2020-07-14 12:25:53 alexmiller Not planning to make CLJ_CONFIG into a path but we may at some point add some way to have additional deps files #2020-07-14 12:40:08 eval2020 good to hear it's being considered, thanks #2020-07-14 15:33:19 seancorfield I would stop relying on CLJ_CONFIG if I could specify one or more additional deps.edn files that went into this slot: <system> <user> <additional file(s) here> <project>. #2020-07-14 16:37:36 alexmiller I prefer that way of thinking about it to path idea above #2020-07-14 16:38:31 alexmiller we had lots of questions the last time we talked through this. Some of those have been resolved since, some have not. #2020-07-14 16:38:42 alexmiller we = Rich and I, that is #2020-07-14 19:42:13 eval2020 The scenarios I sketched could then be solved by specifying an alias or wrapper like clj --deps project-deps.edn ? #2020-07-14 20:06:01 alexmiller well that is the question - how would you want to work with this in practice #2020-07-15 02:16:57 salam is it possible to automatically install clojure cli on jenkins as a custom tool where the jenkins user doesn't have enough privileges (i.e., neither root nor on the sudo list) to do so? #2020-07-15 02:18:16 salam this is what we are getting right now:
[clojure_cli] $ sh -e /var/lib/jenkins/tools/com.cloudbees.jenkins.plugins.customtools.CustomTool/clojure_cli/hudson7631100789357617495.sh
[...]
You need to be root to perform this command.
[...]
Installing libs into /usr/local/lib/clojure
install: cannot create directory /usr/local/lib/clojure: Permission denied
#2020-07-15 02:28:45 seancorfield @abdusalam I believe you can run the script with the --prefix option and a directory to which you have write access. #2020-07-15 02:32:00 seancorfield (yes, I just tested this works) #2020-07-16 19:51:43 markbastian If I want to publish a tools deps project as a jar to a repo (e.g. clojars) and I have an up-to-date pom.xml, what is the "right" artifact to publish if you are using git coordinates for your dependencies? Do I need to create an uberjar and publish that? Is there a way to package the git deps src in a regular jar target? This is assuming a library project with git dependencies. #2020-07-16 20:01:13 alexmiller there is no way to do that #2020-07-16 20:01:26 alexmiller git deps are only usable from source deps #2020-07-16 20:02:21 alexmiller as soon as you say "artifact", you're talking about maven and you need to depend only on other artifacts #2020-07-16 20:03:05 markbastian Makes sense. Seems like an uberjar as lib or jar + git deps as lib would be a mess. #2020-07-16 20:17:51 kenny Is there a Maven command or Clojure lib someone made that uploads a jar file to a S3 bucket? All the tools I've found seem like they expect to build the jar for you. #2020-07-16 20:18:52 alexmiller well you can just do an aws s3 cp if that's sufficient #2020-07-16 20:20:06 kenny I tried that and receive this message from tools-deps "Download corrupted: Checksum validation failed, no checksums available." It actually all works (i.e., jvm starts up with the jar on the cp) but it'd be nice to do it the correct way. #2020-07-16 20:20:33 kenny Or maybe that's a bug in tools-deps? #2020-07-16 20:24:53 alexmiller well if you just upload the jar without the checksum files, you will see this message #2020-07-16 20:26:43 kenny I figured. Is there something out there that can do this the correct way to avoid this warning? #2020-07-16 20:28:13 alexmiller the correct way is to compute an md5 or sha1 checksum for the jar and upload those (and ideally to gpg sign all of those files as well). I do not know of a tool that does that (although I have started working on one) #2020-07-16 20:29:50 alexmiller how that all gets created and uploaded is quite complicated, and clojars does much less (and even less is really required) #2020-07-16 20:30:20 alexmiller really, just the jar and the md5 or sha1 file (and ideally the pom) are probably sufficient #2020-07-16 20:31:04 alexmiller you can use a tool like sha1sum or md5sum to make that #2020-07-16 20:34:16 kenny Wowza. Did not realize what I was getting myself in to. Thank you for all this info. I assume the work you started on is not public yet? #2020-07-16 20:35:49 alexmiller no, nothing usable #2020-07-16 20:37:12 alexmiller I would like to handle a) clojars b) s3 and c) maven central in that order which I think handles both the most common and easiest items first #2020-07-16 20:37:46 alexmiller there are multiple options on each and it is way harder than it should be for sure #2020-07-16 20:42:31 kenny Will try the jar + hash suggestion. And try to restrain myself from building something complete. Thanks again 🙂 #2020-07-16 21:12:25 dominicm Maybe I should build a tool on top of pack. It would force me to finish the api. #2020-07-16 22:11:38 drewverlee For deps that need to be kept in sync, or even in times where its likely two things should udpate together, what strategy have people used?
My first thought would be that deps wouldn't be responsible for this and this more when you need a tool like lein. But if i only have this issue maybe soemthing a tad more lightweight. #2020-07-16 22:31:26 seancorfield @drewverlee I put them next to each other in deps.edn with a comment explaining the relationship. #2020-07-16 22:32:58 seancorfield (it's a project/application concern, not a tooling concern really -- so it's more about making sure people know how/when to change stuff) #2020-07-16 22:34:16 seancorfield BTW, that's also why I'm generally against tools that update my configuration files automatically -- those tools cannot know about such constraints, e.g., I may have very valid reasons for keeping two particular dependencies on older versions together, even if newer versions of one or both exist. #2020-07-16 23:10:09 drewverlee Seems reasonable, I guess I should start with the comment and add more as needed. Thanks @seancorfield #2020-07-17 08:51:18 danieroux @kenny This is how I solved it:
mvn org.apache.maven.plugins:maven-install-plugin:3.0.0-M1:install-file -Dfile=dev-local-0.9.172.jar -DlocalRepositoryPath=s3_m2_repository
aws s3 sync s3_m2_repository
#2020-07-17 15:29:08 kenny Oh, nice! This seems like it'd upload all the necessary files. Sweet. #2020-07-21 02:30:25 kenny This approach still results in the Download corrupted: Checksum validation failed, no checksums available warning message though 😞 #2020-07-17 09:06:39 danieroux @drewverlee I have entries like these:
org.slf4j/slf4j-api {:mvn/version "1.7.14" :required-by :datomic-cloud} #2020-07-17 21:03:38 alexmiller hey clj super fans, there is a new version of clj (and tools.deps) out and I'd love to have some people try it and verify no dep tree differences. you can install the latest dev release with:
brew uninstall clojure
brew install --devel clojure/tools/clojure
#2020-07-17 21:03:38 alexmiller hey clj super fans, there is a new version of clj (and tools.deps) out and I'd love to have some people try it and verify no dep tree differences. you can install the latest dev release with:
brew uninstall clojure
brew install --devel clojure/tools/clojure
#2020-07-17 21:24:41 seancorfield @U064X3EF3 I expect you know about this
Warning: Calling 'devel' blocks in formulae is deprecated! Use 'head' blocks or @-versioned formulae instead.
Please report this issue to the clojure/tools tap (not Homebrew/brew or Homebrew/core), or even better, submit a PR to fix it:
/usr/local/Homebrew/Library/Taps/clojure/homebrew-tools/Formula/clojure.rb:7
#2020-07-17 21:26:02 alexmiller yeah #2020-07-17 21:26:29 alexmiller brew breaks things faster than I can fix them #2020-07-17 21:27:32 seancorfield Then I got
==> Downloading
######################################################################## 100.0%
Error: clojure 1.10.1.536 is already installed
To upgrade to 1.10.1.561, run `brew upgrade clojure/tools/clojure`.
==> `brew cleanup` has not been run in 30 days, running now...
and it did not actually upgrade clojure 😐 #2020-07-17 21:28:16 seancorfield (! 1406)-> clojure -Sdescribe
{:version "1.10.1.547"
:config-files ["/usr/local/Cellar/clojure/1.10.1.547/deps.edn" "/Users/sean/.clojure/deps.edn" ] #2020-07-17 21:28:23 seancorfield Which is not 561 #2020-07-17 21:29:04 alexmiller oh, sorry #2020-07-17 21:29:12 alexmiller brew upgrade --devel clojure/tools/clojure #2020-07-17 21:29:21 seancorfield Nope, that doesn't work either. #2020-07-17 21:29:34 seancorfield Error: invalid option: --devel
#2020-07-17 21:29:37 alexmiller well then brew uninstall clojure first #2020-07-17 21:30:23 alexmiller then install --devel #2020-07-17 21:30:25 seancorfield (! 1413)-> clojure -Sdescribe
{:version "1.10.1.561"
:config-files ["/usr/local/Cellar/clojure/1.10.1.561/deps.edn" "/Users/sean/.clojure/deps.edn" ]
Had to uninstall then install --devel #2020-07-17 21:31:05 alexmiller yeah, that's the correct one #2020-07-17 21:06:58 alexmiller the change of note here is I have rewritten how exclusions are handled if the same dep shows up more than once in the dep tree with different exclusion sets. this is uncommon, but possible and most commonly happens by having the dep with exclusions in one place and with no exclusions in another. in the past, what happened was somewhat arbitrary - depending which you first encountered you may get the dep or not. #2020-07-17 21:08:24 alexmiller the updated behavior is that you will get the intersection of exclusion sets from all occurrences, so in the case above you will always get the dep #2020-07-17 21:09:49 alexmiller in the process of implementing this, I've also substantially refactored the deps expansion code to make it easier to work on and comprehend. while those changes should be a no op, I did change a lot of code and well, you know. #2020-07-17 21:10:17 alexmiller so if anyone has time to do a sanity check, that would be cool #2020-07-17 21:12:59 alexmiller also, in the process of checking these changes, it's become clearer to me that -Stree is likely to lie to you about what happened in exclusion cases like the above, because it's based on the lib map, which is an inverted tree that loses exclusion path info. I'm going to rewrite it in terms of the trace data and then it will be both correct but can also include substantially more useful information (akin to tools.deps.graph trace mode). but going to do other work first. #2020-07-17 21:32:09 seancorfield A quick test shows that it doesn't attempt to download any new dependencies for our monorepo so that looks sane. #2020-07-17 21:33:11 alexmiller :) #2020-07-17 21:34:09 alexmiller having a new clj should make everything stale (so effectively -Sforce), but maybe -Sforce too? #2020-07-17 21:35:27 seancorfield I'm running our script that checks for updated versions of deps -- it builds new projects with our deps and then runs -Stree in those so there no cache. I'll let you know if that turns up any differences. #2020-07-17 21:36:40 seancorfield Looks good. It detected a new version of org.apache.axis/axis and all the other dependencies remained unchanged. #2020-07-17 21:36:50 seancorfield (that script hammers clojure -Stree 🙂 ) #2020-07-17 21:43:26 seancorfield Just for good measure, I blew away the cache folder in every subproject in our monorepo and rebuilt things. No changes in deps as far as I can tell. #2020-07-17 21:45:04 alexmiller Thanks, good to have someone else try it :) #2020-07-17 21:45:52 seancorfield You know me -- always happy to be on the bleeding edge! #2020-07-17 22:06:28 kenny I can try this. @seancorfield how did you check for differences? Set diff of classpath? #2020-07-17 22:16:39 seancorfield Just compared -Stree before and after. #2020-07-17 22:17:33 seancorfield But also ran our test suite to see if any new dependencies got downloaded (none). #2020-07-18 02:24:11 kenny No new deps were printed when running clojure -Spath -Sforce . Also compared the classpath of the old and the new and they are identical. #2020-07-18 02:47:10 alexmiller Thanks, as expected, #2020-07-21 02:31:49 kenny Any idea why clojure -Spom would sometimes result in duplicate entries? e.g., something like this:
...
<build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
</build>
... #2020-07-21 02:32:55 alexmiller are you making a new one or syncing? #2020-07-21 02:33:02 alexmiller and which clj version? #2020-07-21 02:33:20 kenny This resulted from this:
rm -rf pom.xml
clojure -Spom
clojure -Spom
#2020-07-21 02:33:32 alexmiller the answer to your question is certainly that it's a bug :) #2020-07-21 02:33:35 kenny 1.10.1.561 #2020-07-21 02:34:11 kenny This is my deps.edn
{:paths ["src"]
:aliases {:jar {:extra-deps {seancorfield/depstar {:mvn/version "1.0.94"}}
:main-opts ["-m" "hf.depstar.jar" "s3-mvn-upload.jar"]}}} #2020-07-21 02:34:31 alexmiller there was a change to this in the prior version, which I actually have found was bad and a pretty subtle bug. I have a fix for it but haven't gotten it into master yet. #2020-07-21 02:35:02 alexmiller is it ok after the first clojure -Spom ? #2020-07-21 02:35:27 kenny No. It creates this:
<build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
</build> #2020-07-21 02:35:45 kenny Second run:
<build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
<build>
<sourceDirectory>src</sourceDirectory>
</build>
</build> #2020-07-21 02:36:00 alexmiller yeah, I know why it's doing this #2020-07-21 02:36:12 alexmiller well I will try to get a fix out for it soon #2020-07-21 02:36:32 kenny Any idea which version doesn’t have this? 2 versions back? #2020-07-21 02:37:53 alexmiller check the changelog #2020-07-21 02:39:07 alexmiller prior was tools.deps 0.8.677 which corresponds to 1.10.1.510 #2020-07-21 02:39:43 kenny Yeah was looking for the mapping. Is there a place for tdeps -> brew version? #2020-07-21 02:39:59 alexmiller at the top of that changelog is a link to the other changelog #2020-07-21 02:40:05 alexmiller and vice versa #2020-07-21 02:40:23 kenny Perfect. Thank you! #2020-07-21 02:41:08 alexmiller there was a bug in initial gen in the prior version too though so you may not be any happier #2020-07-21 02:42:56 kenny Haha alright. At some point it worked, at least for my simple use case, since I’ve been using -Spom to deploy libs to clojars. Or perhaps I’ve just never noticed it generating something incorrect 😬 #2020-07-21 02:52:31 kenny For those wanting to simply upload a jar file to a S3 Maven repo, I created a small Clojure lib to do (some of) the right things. You can paste it into a bb script if you want it to go a bit faster. https://github.com/kennyjwilli/s3-mvn-upload #2020-07-21 02:54:00 kenny Folks using Datomic’s dev-local may find that useful ^ #2020-07-22 16:50:03 practicalli I'm using https://github.com/seancorfield/clj-new to add a new file/namespace called handlers to an existing project, practicalli/banking. I assume this is the right command given an alias called :new in my user level deps.edn file. It seems to work okay.
clojure -R:new -m clj-new.generate ns=practicalli.handlers
cc @seancorfield #2020-07-22 16:54:09 seancorfield That's a reasonable approach. I'd probably add a :gen alias so I didn't have to type -m clj-new.generate each time. #2020-07-22 16:54:57 seancorfield (caution: the generators are very primitive and not particularly well thought out -- I don't actually find them valuable and I would prefer they not be promoted as a way to do anything!) #2020-07-22 16:55:35 seancorfield At some point, they'll probably be completely rewritten (in a new ns, naturally). #2020-07-22 17:21:27 practicalli Ok, thanks. I only just used it today for the first time successfully. I tried previously but was getting an error as I didnt include any kind of clj-new alias, so content in just getting it working for myself 🙂 #2020-07-22 17:22:10 practicalli I am more interested in writing some templates to use with clj-new, so will focus on that instead, which I assume is quite stable. #2020-07-22 17:31:00 seancorfield @jr0cket Yup, if you run into any problems, feel free to DM me about it. #2020-07-22 17:31:41 seancorfield The built-in app/lib templates are probably a good place to get started. Also let me know if you have suggestions for improving the README around writing templates. #2020-07-22 21:43:01 markbastian Should this be the right invocation? clj -Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}" #2020-07-23 03:06:30 salam a better approach would be to create an alias in the project deps.edn: https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L66
i'm a cursive user myself and this is how i start a socket repl server in the terminal ( clojure -O:socket ) and connect cursive (a socket repl client) to it by creating a remote socket repl run configuration in intellij idea. #2020-07-23 15:26:00 markbastian Nice. Thanks! I think 90% of the answers are in Sean Corfield's deps.edn file. #2020-07-23 15:27:31 salam yup, i personally found a lot of gems in there. 🙂 #2020-07-23 05:48:28 tengstrand I can set up my deps.edn at the root so that it includes a "module" (that has its own deps-edn) like this:
:aliases {:dev {:extra-deps {env/dev {:local/root "environments/dev"}}}
:test {:main-opts ["-m" "polylith.clj.core.cli.poly" "test"]}}
Then I can run:
clj -A:test:dev
...and it will run my tests (that is what the 'test' alias does for me).
But the deps.edn under environments/dev contains its own aliases with some :extra-paths that I want to include also. Is that possible? I imagine that I need to do something like this (which doesn't work):
clj -A:test:dev:dev/local-alias
...where local-alias is defined as an alias in environments/dev/deps.edn:
:aliases {:local-alias {:extra-paths ["../../components/help/src"]}
#2020-07-23 14:34:48 seancorfield No, aliases in dependencies are not relevant. That would be kind of dangerous as you'd suddenly become vulnerable to whatever aliases developers added to their own projects, which could interfere with your own user-level aliases. And you'd also be affected if a dependency switched from Leiningen to Clojure CLI or vice versa -- which runs completely counter to any notion of reproducibility. #2020-07-24 06:04:53 tengstrand Okay, but here I suggest to use existing aliases that lives in environments/dev/deps.edn, that we "import" from deps.edn at the root. Then we don't "mess up things from the outside" just selecting aliases within that "imported" deps.edn. If I "cd environents/dev" first, then I have access to all those aliases which have the purpose to add :extra-paths which I need. But here I have several "sub modules" (not just the one in this example) and wants to do the same from the root deps.edn without having to manually merge all those deps.edn files into one huge deps.edn at the root. #2020-07-24 06:06:49 seancorfield As I said @U1G0HH87L "That would be kind of dangerous". #2020-07-24 06:08:28 seancorfield :local/root isn't special -- it's just another way to identify a dependency. You're asking for dependencies that include deps.edn to have their aliases combined with the current project. That's not a practical way to deal with reproducibility in project builds. #2020-07-24 06:11:44 tengstrand Then I guess I need to solve it with code generation or create my own REPL or similar. #2020-07-24 06:15:43 seancorfield What we do at work, with our 100k+ line monorepo containing 30+ subprojects, is that we have a special subproject (called versions) that contains the "master" deps.edn that has all the aliases needed by all projects, including :defaults which brings in all the :override-deps to pin versions etc, and then all the other subprojects really just have :deps and an alias for :test-deps to add anything needed for testing that subproject (`versions/deps.edn` has a :test alias for all the common testing infrastructure). Then we use CLJ_CONFIG=../versions clojure -...aliases in each subproject. #2020-07-24 06:16:36 seancorfield CLJ_CONFIG lets you treat a specific folder as if it contains your user-level deps.edn (and so any actual ~/.clojure/deps.edn file is ignored -- which is good: the monorepo then control all dependencies and aliases!) #2020-07-29 15:24:31 tengstrand Thanks for the information, can be very useful! #2020-07-23 08:54:01 p-himik There's no way to prevent clj from reading deps.edn in the current directory, is there? #2020-07-23 09:56:20 borkdude @p-himik don't think so. #2020-07-23 09:59:15 p-himik Thanks. That's seems a bit inconvenient because if I want to run some tool with -Sdeps (like Capsule in my particular case) within my project, I cannot really do it without mixing two different classpaths. #2020-07-23 10:02:15 plexus it's fairly annoying, I've had scripts that temporarily move deps.edn out of the way because of this... #2020-07-23 10:04:54 borkdude deps.clj, an unofficial re-implementation of the clj script does have an option to choose a different deps.edn file than the one in the current directory #2020-07-23 10:05:23 vlaaad I think setting :paths in aliases to replace source path from deps.edn is as close as it gets to what you want #2020-07-23 10:06:13 vlaaad and :deps #2020-07-23 10:07:33 p-himik But then I'd have to restructure my whole deps.edn just to make some thirdparty tool happy. That's not good. #2020-07-23 10:08:19 vlaaad not sure I’m following, can you put that third-party tool as a user-level alias? #2020-07-23 10:09:40 p-himik The tool itself - yes. The thing is, it must be run in the presence of deps.edn. But it would be best to not let clj (and I run the tool using clj) also interpret deps.edn. #2020-07-23 10:10:15 p-himik The tool itself is used via -Sdeps, so it's not in the deps.edn. #2020-07-23 10:08:44 borkdude you can also manually construct the classpath (using a script) and pass it to -Scp #2020-07-23 10:08:53 vlaaad that way you don’t touch your project’s deps.edn at all #2020-07-23 10:14:15 p-himik Oh, it's now possible to use :deps in aliases. There's hope! #2020-07-23 10:15:47 borkdude That overrides the base deps? #2020-07-23 10:15:59 p-himik Yep, seems like it. #2020-07-23 10:16:22 p-himik So basically I should be able to mimic an empty deps.edn. Will experiment later today. #2020-07-23 10:16:54 borkdude note that you will still have clojure and spec on the cp by default #2020-07-23 10:17:52 p-himik Hmm, right. But that should've really interfere with tools that are already meant to be run with clj. #2020-07-23 10:27:31 borkdude I also use deps.edn for babashka projects, the extra deps don't hurt, but they can make searching the classpath a tad slower (like a few ms maybe) #2020-07-23 10:29:55 p-himik In my case, they don't anything really bad as well, but I really want to get rid of those slf4j warnings. #2020-07-23 12:27:56 alexmiller yes, using :deps / :paths in an alias is what we call "tooling" and will replace the project :deps/:paths (for the purpose of using a tool that shouldn't use the project classpath) #2020-07-23 12:28:43 alexmiller until yesterday, you could only use those via -A, but as of the latest dev release yesterday, tooling -T is now a thing #2020-07-23 12:38:02 p-himik Awesome, thanks! #2020-07-23 12:40:41 alexmiller more to come on that and other changes in latest, just been kind of busy with ... other things :) #2020-07-23 13:06:54 Tamizhvendan S Hi, I am getting FileAlreadyExistsException while trying to build an uberjar
Here is my deps.edn file
{:paths ["src" "resources"]
:deps {io.appium/java-client {:mvn/version "7.3.0"}
org.clojure/data.xml {:mvn/version "0.0.8"}
metosin/reitit-ring {:mvn/version "0.5.1"}
metosin/reitit-middleware {:mvn/version "0.5.1"}
metosin/reitit-spec {:mvn/version "0.5.1"}
mount {:mvn/version "0.1.16"}
ring/ring-jetty-adapter {:mvn/version "1.7.1"}
org.slf4j/slf4j-simple {:mvn/version "1.7.30"}}
:aliases {:dev {:extra-paths ["dev"]
:extra-deps {org.clojure/tools.namespace {:mvn/version "1.0.0"}}}
:uberjar {:extra-deps {seancorfield/depstar {:mvn/version "1.0.94"}}
:main-opts ["-m" "hf.depstar.uberjar"]}}} #2020-07-23 13:07:49 Tamizhvendan S Here is the command that I am running clj -A:uberjar my-app.jar #2020-07-23 13:08:17 Tamizhvendan S Building uber jar: my-app.jar
Execution error (FileAlreadyExistsException) at com.sun.nio.zipfs.ZipFileSystem/createDirectory (ZipFileSystem.java:422).
META-INF/versions/9/
#2020-07-23 13:09:19 Tamizhvendan S Am I missing anything obvious here? #2020-07-23 17:19:37 cap10morgan What are folks using to store / set their project's version number so it makes it into the generated pom.xml (in an automated way; think CD pipeline, not hand-editing the pom.xml)? #2020-07-23 17:31:23 seancorfield I'm treating the pom.xml as the "system of record" for the version number and deriving it elsewhere from the POM file. #2020-07-23 17:33:16 seancorfield (and that's the workflow that I use for all my deps.edn-based projects that get deployed to clojars -- note that Contrib libs such as core.cache, core.memoize, tools.cli that I maintain already follow this workflow with "Maven Release" process in CI editing the pom.xml to change the version number explicitly based on the version you enter in that Jenkins form) #2020-07-23 19:53:22 cap10morgan @seancorfield OK, thanks. What tools do you use to read / bump it in the POM file? Curious if you have anything resembling the lein release workflow going there. #2020-07-23 22:35:28 slimslenderslacks hey @cap10morgan, we're treating git tags as the source of truth for versions. When we deploy, the version in the generated pom.xml is made to be in sync with "git describe". Since the pom.xml is only ever needed for the deploy action, the pom.xml doesn't actually have to be checked in. #2020-07-23 22:42:34 seancorfield Note that for http://clojars.org and also for http://cljdoc.org there's an expectation that pom.xml contains a lot more than just the dependencies and the version, so having it as a "loose" file outside git would be... ill-advised if the generation process doesn't include all that information (which lein does put there, but clojure -Spom does not, so I'm talking about deps.edn projects -- note that clj -A:new would generate initial pom.xml files with all the appropriate fields for http://clojars.org and http://cljdoc.org) #2020-07-23 22:51:34 slimslenderslacks Ya, our process is still using lein for the pom.xml generation. That's super interesting @seancorfield - I was probably going to hit this in the near term as I had been planning on migrating to use clojure -Spom. Thanks! #2020-07-23 22:53:34 seancorfield clojure -Spom is fine for updating <dependencies> in an existing pom.xml file but it only generates a very bare-bones version if no file exists. That's why I added a full-featured pom.xml to the templates that clj-new uses for app, lib, and template. #2020-07-23 22:57:44 slimslenderslacks When git describe is the source of truth for version maybe we'd compose clojure -Spom to update the dependencies, with a git-describe->pom-version , and keep a clj-new generated pom.xml checked in. #2020-07-23 23:01:47 slimslenderslacks It sounds like a checked in pom.xml might currently be the best place to version the extra metadata that clj-new adds. #2020-07-23 23:05:10 seancorfield clj-new creates a baseline pom.xml. It never updates it after that. #2020-07-23 23:06:52 seancorfield clojure -Spom updates an existing pom.xml with the latest dependencies (from deps.edn) so as long as you always do it prior to a build/deploy cycle, I guess those changes don't really matter. #2020-07-23 23:07:23 seancorfield Your git describe update also seems to be "do it before build/deploy" but don't really care about the changes. #2020-07-23 23:08:06 seancorfield However, a lot of people would be very confused by a pom.xml file in a repo that didn't match the dependencies in deps.edn and/or didn't seem to reflect the current released version 🙂 #2020-07-23 23:08:35 seancorfield So whether you check it in or not after those changes probably depends on whether it's going into a public repo I suppose 🙂 #2020-07-23 23:12:24 slimslenderslacks Ya, I agree. It could be confusing to see a pom.xml with a weird version. You'd almost want the checked in copy to have a version of :project-uses-git-tags , and maybe have an empty dependencies, so it appears to be there just to define the extra maven stuff that you can't derive from elsewhere. #2020-07-23 23:13:54 slimslenderslacks I don't mind having a file with the correct version, I was more just trying to avoid the commits that go along with those workflows. #2020-07-23 23:14:32 slimslenderslacks Our unit tests and commit messages are the signals the produce Tags, and Tags are the signals that create new publishes to maven repos. #2020-07-23 23:15:21 slimslenderslacks (from a CI/CD perspective) #2020-07-23 23:23:43 seancorfield Yup. I think internal CI/CD processes are under different constraints and, unless you are publishing every successful build to (public) Maven Central / Clojars and also publishing every version to a (public) GitHub repo or similar, for internal releases you can definitely just keep a baseline pom.xml under version control and update that as needed for each build to your repo (and throw it away afterward). #2020-07-23 20:00:20 alexmiller there are maven plugins for that although it's all pretty tedious #2020-07-23 20:01:05 alexmiller the mvn release / deploy infrastructure does a lot for you but you have to buy pretty far into it all the way for it to be worth doing #2020-07-23 20:01:20 alexmiller (but it's a lot less fragile than the lein release stuff) #2020-07-23 20:02:53 cap10morgan Thanks Alex. lein release can indeed be fragile. 🙂 #2020-07-23 20:12:44 alexmiller the idea to put the version inside a file was probably bad from the beginning #2020-07-23 20:13:50 seancorfield @cap10morgan For my clojars-deployed stuff, I just update it as part of my general release process where I'm updating the README and the CHANGELOG and I update both the <version> and the <tag> fields in the pom.xml file all at the same time. #2020-07-23 20:14:57 seancorfield (so I grep for <old-version>|<new-version> so I can see all the places that refer to the current release and what will be the next one, and then I can visually verify that all references get changed) #2020-07-23 20:15:52 seancorfield In most of my libs, that includes several places in the README and usually in the documentation files as well that refer to installation/getting started and when new features were introduced. #2020-07-23 20:19:06 alexmiller I've automated that step on some libs :) #2020-07-23 20:21:07 seancorfield I still have artisanal documentation 🙂 #2020-07-24 15:07:35 drewverlee Given a deps in project A that has a :local/root to another deps in project B. If i update a dep in project B and start my repl in poject A i would expect to have the new dep from project B be used. This doesn't seem to be the case. Any ideas what could cause this? #2020-07-24 15:11:46 alexmiller yes, you need to -Sforce to recompute the classpath #2020-07-24 15:12:01 alexmiller right now, changes in transitive local deps are not automatically discovered #2020-07-24 15:12:57 drewverlee gotcha. thanks alex #2020-07-24 15:15:35 borkdude I saw a mention of -T in this channel. Where is this documented? #2020-07-24 15:23:55 alexmiller in not yet published docs :) #2020-07-24 15:24:17 alexmiller only exists in the newest dev release of clj #2020-07-24 15:24:34 alexmiller so I wouldn't tell anyone to use it yet #2020-07-24 15:24:59 alexmiller but it's exactly what's supported now via -A for :deps and :paths in aliases, it just has an independent name now #2020-07-24 15:58:36 borkdude I think I'll just wait for the docs 🙂 #2020-07-24 16:02:04 alexmiller it's not really giving you anything new that you didn't already have :) #2020-07-24 16:05:07 alexmiller there are other new and interesting things in this release though :) #2020-07-24 16:06:48 vlaaad Now I'll have to go through the changes... #2020-07-24 16:07:15 alexmiller writeup is coming #2020-07-26 16:37:26 kenny Something strange is going on with s3 deps and tools-deps. I’m on 1.10.1.561. Given this deps.edn,
{:deps {com.datomic/dev-local {:mvn/version "0.9.184"}
com.datomic/client-cloud {:mvn/version "0.8.102"}}
:mvn/repos {"kwill-mvn" {:url ""}}}
the output of clojure -Stree -Sforce is
org.clojure/clojure 1.10.1
org.clojure/core.specs.alpha 0.2.44
org.clojure/spec.alpha 0.2.176
com.datomic/dev-local 0.9.184
com.datomic/client-cloud 0.8.102
com.cognitect/anomalies 0.1.12
com.datomic/client 0.8.111
...
It is not picking up the dev-local deps specified in the jar’s pom.xml. If I change the dev-local dep to be a local/root pointing to the Maven jar, I get the correct tree.
{:deps {com.datomic/dev-local {:local/root "/Users/kenny/.m2/repository/com/datomic/dev-local/0.9.184/dev-local-0.9.184.jar"}
com.datomic/client-cloud {:mvn/version "0.8.102"}}
:mvn/repos {"kwill-mvn" {:url ""}}}
Output of clojure -Stree -Sforce
org.clojure/clojure 1.10.1
org.clojure/core.specs.alpha 0.2.44
org.clojure/spec.alpha 0.2.176
com.datomic/dev-local /Users/kenny/.m2/repository/com/datomic/dev-local/0.9.184/dev-local-0.9.184.jar
org.clojure/tools.analyzer 0.6.9
com.google.errorprone/error_prone_annotations 2.3.4
...
Note how dev-local is now bringing in deps and previously it was not. #2020-07-26 17:32:35 hiredman If I recall, maven doesn't use the pom out of a jar, it keeps the pom alongside the jar, so it looks like you are deploying your jar to your maven repo with a different pom then you are building into it #2020-07-26 17:44:49 kenny Ah, that does make sense. From a previous discussion with Alex, I was under the impression that I only needed to upload a jar & a hash to s3 to get it to work. #2020-07-26 18:43:28 hiredman That may be the minimal required for maven to download the jar, but more may be required to communicate dependencies #2020-07-26 22:45:34 alexmiller I think you may actually just be seeing bad output from -Stree #2020-07-26 22:47:06 alexmiller in general, -Stree will only list a dep once (even when it's included under multiple branches) - and it's pretty arbitrary where you will see it. for these two libs there are many shared deps. #2020-07-26 22:49:54 alexmiller these two deps particularly share deps but with different exclusion sets, a case where this version of clj newly makes better, more intentional and reproducible choices. Unfortunately, -Stree is built from the lib map, which does not retain the original path structure and it's quite likely to see deps incorrectly under exclude points (because they are included at other places). -Stree is actively misleading in these cases. (the actual set of libs is right though!) #2020-07-26 22:51:31 alexmiller what I'm trying to say is you may have seen two different things from -Stree, and yet they are actually both right in terms of the transitive lib set, but wrong in how they are showing you the tree #2020-07-27 15:20:21 kenny @alexmiller While that may be happening, the S3 Maven dep approach would result in a classnotfound exception when loading dev-local in the REPL (something with fressian). Switching to local/root with the jar does not result in the classnotfound. I also uploaded a pom to S3 and that approach did not result in a classnotfound. #2020-07-27 16:01:52 alexmiller what jar are you talking about wrt s3? #2020-07-27 16:02:00 alexmiller dev-local? #2020-07-27 16:02:32 alexmiller if so, what you're saying makes sense #2020-07-27 16:03:26 alexmiller afaik, maven will use the pom to report deps (which is a layer that clj is going through for that) #2020-07-27 16:04:13 kenny Yes - dev-local. #2020-07-27 16:05:49 kenny To be clear, here's what works and doesn't work.
S3 jar - classnotfound
S3 jar, jar md5 - classnotfound
S3 jar, jar md5, pom - works
local/root jar - works #2020-07-27 17:23:18 alexmiller yeah, that makes sense to me. sorry I didn't understand what you were doing earlier #2020-07-27 17:23:31 alexmiller local/root jar will use the pom inside the jar #2020-07-28 03:12:09 olivergeorge I'm curious how RELEASE resolves in this maven dep...
{:extra-deps {clj-kondo {:mvn/version "RELEASE"}}
:main-opts ["-m" "clj-kondo.main"]}
Perhaps it's a normal maven feature. Hard to google for "release"! #2020-07-28 03:26:56 alexmiller RELEASE (and LATEST) are special "virtual" version numbers that tell Maven to look for the newest released version (non snapshot) #2020-07-28 03:27:30 alexmiller they are officially unsupported by clj (but do kind of work with some big caveats) #2020-07-28 03:28:08 alexmiller big caveat #1 - they break caching and you will never see a new release after the one that it first resolves to unless you -Sforce (or otherwise cause a cache recompute) #2020-07-28 03:29:50 alexmiller at some point, we may actually prevent caching at all if you use these "versions" #2020-07-28 03:30:42 alexmiller subtle caveat #2 - the answer you get is only as good as the metadata index in the repository. for maven or clojars, no worries. if you're using a bespoke s3 repo, then you probably aren't updating the right metadata files and you usually won't see the newest version #2020-07-28 03:32:04 alexmiller generally, I think it's harmless to do this in -Sdeps to grab "latest". it's mildly bad to do it for tools in your deps.edn aliases (like the example here). it's very bad to do it for your project deps. #2020-07-28 05:21:53 pmonks Does there happen to be an equivalent for git/sha coordinates? That could be quite handy… #2020-07-28 06:01:35 alexmiller Intentionally not, for the same reasons re caching #2020-07-28 06:04:59 pmonks I guess I don’t understand the caching issue. If you resolve “RELEASE” to a concrete version, and that version happens to exist in the cache already, wouldn’t tools.deps use it? #2020-07-28 06:05:28 pmonks (and conversely, if it doesn’t, wouldn’t that version be added to the cache, and thereby eligible for future “cache hits”) #2020-07-28 06:06:15 alexmiller The caching is based on time stamps of deps,edn files. If the file never changes (set to RELEASE), the class path is never recomputed, even if newer versions come out #2020-07-28 06:06:57 alexmiller Caching of classpath (not lib version) #2020-07-28 06:08:24 pmonks Ah ok. I assume there’s a good reason for making the deps.edn file the cache key, rather than just recomputing dependencies the classpath every time and having those in the cache? #2020-07-28 06:09:17 alexmiller Recomputing deps starts a second JVM so it’s at least a second, usually more depending on io #2020-07-28 06:09:53 alexmiller This is why clj starts faster than lein usually #2020-07-28 06:10:15 alexmiller It also goes to build repeatability #2020-07-28 06:11:29 alexmiller Or “clj repeatability” #2020-07-28 06:17:04 pmonks Ok. I’ll assume there’s a good reason for needing a second JVM to do that (not requesting an explanation, btw, unless you’d like to provide one), and understood full well the performance implications of starting a JVM. #2020-07-28 06:41:24 alexmiller it's running a Clojure program with a different classpath the program you're executing #2020-07-28 06:43:23 alexmiller specifically, it's running tools.deps to compute the classpath based on your deps.edn (talking to Maven, maybe S3, maybe Git, etc). in the end it spits out the results in cache. Your actual program is then started with the classpath you intended. Once we've cached that work, we can spit the first part and just run the program. #2020-07-28 06:52:28 pmonks So the advantage of caching based on the deps.edn file, rather than the classpath it contains, is all the I/O to talk to Maven, S3, git, etc.? #2020-07-28 07:09:04 alexmiller well, that's all of the work. not much point to caching otherwise #2020-07-28 07:11:21 alexmiller expanding and downloading a full deps tree can easily mean dozens of jars and many megabytes of network traffic (if your m2 and/or git cache is cold) #2020-07-28 07:16:04 alexmiller even if you have all the jars, expanding the tree is more than you'd want to wait, example in tools.deps.alpha repo (~80 jars in the full tree), all jars in cache so no downloads: #2020-07-28 07:16:22 alexmiller amac:tools.deps.alpha alex$ time clj -Sforce -e nil
real 0m3.028s
user 0m12.567s
sys 0m0.756s
amac:tools.deps.alpha alex$ time clj -e nil
real 0m0.745s
user 0m1.647s
sys 0m0.133s
#2020-07-28 07:17:39 alexmiller (one caveat here is that the clj uber jar is not currently aot compiled - that's coming soon and will improve that first time, which includes a lot of loading) #2020-07-28 07:18:22 alexmiller but it's still not time anyone wants to wait #2020-07-28 07:20:51 alexmiller with a cold m2 cache, time is about 7.5 s (aot may knock a couple seconds off that) #2020-07-28 15:00:22 pmonks I can see why that approach will be faster, though it breaks down (cache becomes stale) in the presence of dependencies that are expressed using a “floating” version reference (RELEASE, LATEST, SNAPSHOT, etc.).
Those were deprecated in Maven3, and perhaps tools.deps should follow suit and emit a warning when it encounters such a thing? #2020-07-28 15:00:46 alexmiller perhaps #2020-07-28 05:24:33 seancorfield @pmonks My dot-clojure deps.edn shows how you can resolve the head of the default branch -- in the comments. #2020-07-28 05:26:00 seancorfield And then (load-master 'clojure/tools.trace) #2020-07-28 05:26:36 seancorfield (following the https://github.com URL pattern, not the org.clojure/tools.trace Maven artifact pattern) #2020-07-28 05:41:29 pmonks Potentially dumb question, but is it expected that code inside an EDN file would be executed? Or does that rely on a coincidence of how the Clojure EDN reader functions? #2020-07-28 05:53:23 seancorfield @pmonks Can you give a specific example? #2020-07-28 05:54:01 seancorfield I mean, EDN can contain tagged literals and you can provide reader functions that process those... but that's not exactly "code inside an EDN file". #2020-07-28 05:58:57 alexmiller edn files are data, not code, and are not evaluated #2020-07-28 06:00:33 alexmiller They are read but that will only construct tagged literals, and clj does not provide a way to install readers for tagged literals #2020-07-28 06:03:04 pmonks @seancorfield your example includes code (`(require '[clojure.tools.gitlib :as gitlabs]) …`) - how is that code evaluated? #2020-07-28 06:06:40 seancorfield Those are comments showing what you would execute in your REPL... #2020-07-28 06:07:03 seancorfield The files contains lots of comments explaining how to use the aliases, with links to repos. #2020-07-28 06:09:30 pmonks Oh ok. So I would not put those in a deps.edn then. #2020-07-28 06:09:52 seancorfield Huh? Those are comments and they are in a deps.edn file. #2020-07-28 06:10:16 pmonks Yes they are, and as a result I would expect to uncomment them and have them Just Work:tm: . #2020-07-28 06:10:23 seancorfield I mean, that repo literally is my ~/.clojure/deps.edn file. #2020-07-28 06:10:36 pmonks But if I’m following you, you’re saying that’s not correct - that those lines need to be copypasta’ed into a REPL. #2020-07-28 06:11:03 seancorfield Forgive me but I'm a bit shocked you would think code in EDN files would be evaluated... #2020-07-28 06:11:14 pmonks I didn’t - that’s why I asked the question. #2020-07-28 06:11:48 pmonks I was quite surprised to see commented out code in an EDN file. #2020-07-28 06:12:26 pmonks (and by “quite”, I mean “very”…) #2020-07-28 06:13:37 pmonks Regardless, @alexmiller separately explained that by design you can’t do an equivalent of RELEASE for git/sha coords in a deps.edn file. #2020-07-28 06:14:34 pmonks Why would you assume I’ve read some random deps.edn file? Sadly I don’t have that much disposable time… #2020-07-28 06:17:03 seancorfield Well, you read my file and came to a ... strange conclusion ... so I offered another example deps.edn and wondered what conclusion you would come to from that... #2020-07-28 06:18:58 pmonks In general, answering questions people didn’t ask is a wonderful source of confusion. 😉 #2020-07-28 06:20:36 seancorfield With the Clojure CLI/`deps.edn` the "equivalent" to Maven/Leiningen/etc is often "some code that you run" -- which seemed like an acceptable answer to your question. #2020-07-28 06:21:31 seancorfield That's why I was so puzzled by your follow-up question, as it seemed unrelated to my answer (I was thinking "No, of course reading EDN won't execute code!"). #2020-07-28 06:22:37 seancorfield I guess I should add "Here's some code you can run in your REPL" to my deps.edn file where there are code examples 🙂 #2020-07-28 06:23:04 pmonks That would have avoided any confusion, no doubt. 😉 #2020-07-28 06:24:20 seancorfield It is sometimes so hard to predict your audience's state of mind 😆 (when writing documentation, I should add). #2020-07-28 06:27:40 pmonks Professional technical writing is such an under-appreciated skill, I’ve found. #2020-07-28 07:22:14 alexmiller no, that's not a built-in feature of Clojure (or Java). there are ways to do some variants of this but it's not common #2020-07-28 07:22:28 alexmiller well, do you mean local dir or local jar? #2020-07-28 07:23:02 alexmiller if local dir, you can just use Clojure's normal :reload or :reload-all mechanism built into require #2020-07-28 07:23:24 spf i meant local dir #2020-07-28 07:23:37 alexmiller yeah, so that's just normal reloading #2020-07-28 07:23:45 spf ok. cool. thx #2020-07-28 13:06:18 rickmoynihan any idea why running brew upgrade clojure/tools/clojure installs 1.10.1.561 not 1.10.1.590 ? #2020-07-28 13:28:48 alexmiller 561 is the newest stable release #2020-07-28 13:28:58 alexmiller 590 is a dev release #2020-07-28 16:10:48 seancorfield Following up from my comment in the announce thread:
{:paths [:clj-paths :resource-paths]
:aliases
{:clj-paths ["src/clj" "src/cljc"]
:resource-paths ["resources"]}}
Is that correct, or should :clj-paths and :resource-paths be underneath an alias? #2020-07-28 16:11:24 seancorfield If that is correct, I don't see the value since they are fixed labels and values -- am I missing something @alexmiller? #2020-07-28 16:12:28 alexmiller oh, that is right - they are aliases #2020-07-28 16:12:40 alexmiller an alias is just a name for some data #2020-07-28 16:13:37 alexmiller this was the original meaning here but we have kind of gotten off talking about things that way by tying it into clj's usage of alias data to affect classpath construction #2020-07-28 16:14:11 seancorfield OK, so how is that useful since they are just fixed values? #2020-07-28 16:14:20 alexmiller at the moment, it's not very useful #2020-07-28 16:14:33 seancorfield Also, you have this paragraph twice "Replace project environment ("tool")" -- one with plenty of detail, one with very little. #2020-07-28 16:16:01 alexmiller that's actually intentional - the first part is kind of an overview, and the second is more how the tool works with more detail #2020-07-28 16:16:35 alexmiller doing a lot of heavy editing on this page lately but will likely change more over time #2020-07-28 16:17:13 seancorfield OK, I think it's just really confusing at the moment with two identical headers... #2020-07-28 16:17:38 alexmiller I'm ok with that for the moment :) #2020-07-28 16:17:55 alexmiller the toc at the top side bar may make the structure more evident #2020-07-28 16:18:25 alexmiller I've gone through about 5 different structures for this page recently, and I don't think I'm done #2020-07-28 16:24:05 alexmiller yeah, I've been fighting that, haven't decided what to do about it yet #2020-07-28 16:24:32 alexmiller not enough words :) #2020-07-28 16:24:40 alexmiller it's like naming things is hard #2020-07-28 16:28:10 seancorfield Tools are just so useful 🙂 #2020-07-28 16:14:36 alexmiller but if you had say, a build tool that wanted to also use a subset of paths it might be useful #2020-07-28 16:14:42 alexmiller ;) #2020-07-28 16:17:37 seancorfield The Maven install example is missing an alias tho' (even if the paths one isn't), yes?
{:aliases
{:fn clojure.tools.deps.alpha.install/install
;; :args map could be provided but can pass on command line instead
}} #2020-07-28 16:18:28 seancorfield That's meant to be executed with -X:some-alias (most people will pick -X:install I suspect) #2020-07-28 16:18:44 alexmiller ah, yes that is #2020-07-28 16:18:53 alexmiller I actually saw that late last night and forgot to fix #2020-07-28 16:20:39 seancorfield I'm curious as to why that ability surfaced as a built-in option, given that JAR files can already be used as :local/root deps? #2020-07-28 16:21:24 alexmiller it's not a built-in option, it's a built-in program #2020-07-28 16:22:01 alexmiller didn't seem big enough to make into its own thing, depended heavily on all the code in tools.deps, wanted it for dev-local, etc #2020-07-28 16:22:16 seancorfield Interesting... #2020-07-28 16:22:23 alexmiller serves as a good example :) #2020-07-28 16:22:59 alexmiller the plan is to shift some of the magic switches in clj over to just be programs you can -X #2020-07-28 16:23:33 alexmiller some of those might become built-in to the root deps.edn, not sure yet #2020-07-28 16:30:02 seancorfield The install line you gave in the announcement doesn't seem to work
Error: No available formula with the name "clojure/tools/
#2020-07-28 16:30:23 seancorfield That's from brew install clojure/tools/ #2020-07-28 16:30:31 seancorfield (after a brew uninstall clojure) #2020-07-28 16:33:16 seancorfield Should be 1.10.1.590 #2020-07-28 16:33:29 alexmiller Ah, shoot #2020-07-28 16:34:41 alexmiller Edited #2020-07-28 16:54:37 rickmoynihan wow need to get my head around these new tools.deps features; they might make some stuff I’ve been planning to do completely redundant 🤔 #2020-07-28 16:58:11 rickmoynihan how is the -T alias different to having an alias with a :deps key? #2020-07-28 16:58:43 alexmiller It’s not #2020-07-28 16:58:57 seancorfield I need to figure out how to get the clojure script updated on a couple of our older servers since this would help eliminate some of the extra script stuff we've written (but I think there's a blocker for installing clojure on a couple of our old servers). #2020-07-28 16:59:13 alexmiller It is exactly what was already there via -A, just has its own switch now #2020-07-28 17:00:38 rickmoynihan ok so it just makes the intent clearer on the command line? #2020-07-28 17:01:03 alexmiller Yep #2020-07-28 17:01:51 alexmiller A is meant to be an “all” kind of thing but this didn’t have its own name, so now it does #2020-07-28 17:02:48 rickmoynihan so what would -T do with :extra-deps instead of :deps? #2020-07-28 17:03:02 alexmiller (Which also helped further clarify some things in the code which has gone through some substantial changes) #2020-07-28 17:03:12 alexmiller It would be ignored #2020-07-28 17:04:08 rickmoynihan wow I had no idea how many deps had unqualified names. I thought it was just a handful, but it’s way more than I expected #2020-07-28 17:04:20 alexmiller Each of the sub processes takes what it expects and ignores the rest, like all good Clojure programs :) #2020-07-28 17:06:34 seancorfield Should this check include tools_aliases now?
-h|--help|"-?")
if [[ ${#main_aliases[@]} -gt 0 ]] || [[ ${#all_aliases[@]} -gt 0 ]]; then
break
else
help=true
shift
fi
;; #2020-07-28 17:08:10 alexmiller No? #2020-07-28 17:09:55 seancorfield Maybe I'm just missing the actual flow of the script then, nm. #2020-07-28 17:10:30 alexmiller Yeah this has to do with falsely interpreting a -h appended to main aliases as a help opt #2020-07-28 17:10:30 seancorfield (I was just grepping for (tool|all).alias through the new script out of curiosity) #2020-07-28 17:11:52 alexmiller Would be curious if you try the win install :) #2020-07-28 17:13:01 seancorfield @alexmiller re: -h -- that means that clojure -A:test:runner -h and clojure -T:test:runner -h behave differently. #2020-07-28 17:14:10 seancorfield (! 1537)-> clojure -T:test:runner -h|more
Version: 1.10.1.590
...
(! 1538)-> clojure -A:test:runner -h|more
Unknown option: "-h"
Wasn't sure whether that was intentional or not since you said -T and -A were "the same"? #2020-07-28 17:15:38 seancorfield (the first case displays help -- -h is not passed to the main program; the second case passes -h to the main program in the test runner here) #2020-07-28 17:27:39 alexmiller this is kind of a corner case - the condition is guarding against having -A/-M supplying the first part of the main args and then -h on the command line where it is not actually the main args to pass to clojure.main. I'm not worried about it. #2020-07-28 17:40:24 seancorfield Fair enough. Just wanted to point out that it is a difference between -A and -T 🙂 #2020-07-28 17:57:25 alexmiller there's no difference in how :deps and :paths work, which is what I meant #2020-07-28 17:24:26 rickmoynihan Just tried doing -X:myfn in one of my projects and I get the error: #2020-07-28 17:24:28 rickmoynihan clojure -X:myfn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath.
#2020-07-28 17:25:45 alexmiller are you on clj 1.10.1.590? #2020-07-28 17:26:16 rickmoynihan I thought I was #2020-07-28 17:26:34 rickmoynihan yep #2020-07-28 17:26:44 rickmoynihan $ clojure --help
Version: 1.10.1.590
...
#2020-07-28 17:30:50 seancorfield I get that too, also on 1.10.1.590:
(! 974)-> clj -Sdescribe
{:version "1.10.1.590"
:config-files ["/usr/local/Cellar/ #2020-07-28 17:30:52 alexmiller if you clj -Sdescribe and then jar tf <the-install-dir/clojure-tools-1.10.1.590.jar | grep exec.clj #2020-07-28 17:30:58 alexmiller do you see it? #2020-07-28 17:32:32 alexmiller what installer are you using? #2020-07-28 17:32:51 rickmoynihan the jar is further down than that in /usr/local/Cellar/ #2020-07-28 17:33:09 alexmiller sorry, bad multi copy/paste on my part, that's right #2020-07-28 17:33:16 rickmoynihan exec.clj is in there yes #2020-07-28 17:33:28 alexmiller this is mac? linux? #2020-07-28 17:33:34 rickmoynihan $ jar tf /usr/local/Cellar/
#2020-07-28 17:33:47 rickmoynihan mac #2020-07-28 17:33:58 seancorfield (yup, same here) #2020-07-28 17:33:59 rickmoynihan brew was the installer #2020-07-28 17:34:25 alexmiller clj -Sdeps '{:aliases {:myfn {:fn clojure.core/pr}}}' -X:myfn is a standalone example #2020-07-28 17:34:46 rickmoynihan edn is broken #2020-07-28 17:34:55 seancorfield Fails for me #2020-07-28 17:35:07 alexmiller edn is broken means? #2020-07-28 17:35:17 rickmoynihan ah wait sorry was in a project with a broken edn file #2020-07-28 17:35:23 seancorfield (! 982)-> clj -Sdeps '{:aliases {:myfn {:fn clojure.core/pr}}}' -X:myfn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath.
Full report at:
/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/clojure-12219293780956477884.edn #2020-07-28 17:35:36 alexmiller ah, I was able to repro #2020-07-28 17:35:41 rickmoynihan $ clj -Sdeps '{:aliases {:myfn {:fn clojure.core/pr}}}' -X:myfn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath.
Full report at:
/var/folders/ln/536xqskd3_g80n5pdbsjkrjw0000gn/T/clojure-8308868078358895829.edn #2020-07-28 17:35:43 rickmoynihan too #2020-07-28 17:35:54 alexmiller it matters whether you're in a directory with a deps.edn #2020-07-28 17:36:17 alexmiller well, that's definitely a bug :) #2020-07-28 17:36:20 alexmiller will fix #2020-07-28 17:36:24 alexmiller thx #2020-07-28 17:36:38 seancorfield Crashes for me whether there's a deps.edn file or not. #2020-07-28 17:36:44 rickmoynihan yeah same #2020-07-28 17:36:57 alexmiller hm, maybe it's a cached classpath for me or something #2020-07-28 17:37:20 rickmoynihan there are 3 hard things in computer science… #2020-07-28 17:37:52 seancorfield I tried it via an alias in a deps.edn file (shown above), then tried via the command line with just {} in deps.edn, and then tried in a directory with no deps.edn -- all of them crash the same way. Even tried -Sforce and it still failed. #2020-07-28 17:38:35 rickmoynihan I tried two of those; just tried with -Sforce too — and same #2020-07-28 17:38:46 alexmiller well, that's weird. :) I've been using this for a while and others have tested it as well, somehow never saw an error #2020-07-28 17:39:13 alexmiller I'm sure it will make sense when I understand why it's doing it ... this is why it's a dev release! #2020-07-28 17:40:45 alexmiller -X has to go at the end, in case that affects anything you tried #2020-07-28 17:41:16 rickmoynihan nope, I had -X as my only alias #2020-07-28 17:42:38 rickmoynihan Well I’d be happy to try and help narrow debug it but I’m afraid I need to call it an evening 😞
I’ll check back here tomorrow.
Thanks for your help, and also thanks for the feature… it looks like it’ll be really useful. #2020-07-28 17:43:56 alexmiller ah, this makes sense to me now, and why it worked for me #2020-07-28 17:44:29 alexmiller and why things others tried worked. such a combination of coincidences... anyhow will fix #2020-07-28 17:45:47 seancorfield Now I'm curious, what's the bug? #2020-07-28 17:46:23 alexmiller it's not putting that class on the classpath, obviously :) #2020-07-28 17:46:32 alexmiller I was testing in the tools.deps.alpha repository itself #2020-07-28 17:46:38 alexmiller where it is #2020-07-28 17:47:01 alexmiller and others were testing things that used the install program, which requires you to pull in tools.deps.alpha #2020-07-28 17:47:15 alexmiller so those all worked #2020-07-28 17:47:27 alexmiller I may actually have flubbed a merge somewhere #2020-07-28 19:00:31 alexmiller @seancorfield @rickmoynihan ok, new clj 1.10.1.596 is available, should fix this #2020-07-28 22:39:58 rickmoynihan works now! Thanks 🙂 #2020-07-28 23:10:25 alexmiller thx #2020-07-28 19:05:37 seancorfield Looks fixed! Thank you! #2020-07-28 19:08:47 alexmiller if you try it on windows (or linux), would be happy to hear there too #2020-07-28 19:09:49 seancorfield @vlaaad If you want to get strings from the command line in your -X-executed function, you can always use *command-line-arg* (although you'd have to discard leading arguments passed to the CLI itself I guess) #2020-07-28 19:11:37 seancorfield (! 1010)-> clj -A:test -X:example :test value
("-X:example" ":test" "value") ({:test value})
Looks like you'd only have to skip over the first arg? #2020-07-28 19:12:50 vlaaad I don't want to make every arg a string 🙂 #2020-07-28 19:14:21 seancorfield How would you want the CLI to distinguish between "I want this as a string" vs "I want this as an EDN value"? #2020-07-28 19:17:09 seancorfield It seems like you'd have to try to read as EDN and if you got an error or a symbol then use the whole argument as a string, else whatever the EDN reader produced? Which is open to all sorts of surprises I would expect...? #2020-07-28 19:20:12 vlaaad I guess it's impossible to have it both simple and easy 😞 #2020-07-28 19:24:34 vlaaad I don't want to deal with escaping because I use both powershell and bash that have different escaping rules. With those, whenever I have a need to escape I wish I just used a file. This might be a bit too limiting for convenience, but I'd be fine with this:
everything that can be read as number/boolean/keyword/nil -> number/boolean/keyword/nil
everything else, including things that can be read as collections -> string #2020-07-28 19:27:09 vlaaad it doesn't work with repeated args though #2020-07-28 19:32:55 vlaaad I've been thinking a bit more about simple vs easy, and currently I think I'm overreacting, I think escaping might be an okay price to pay for the limitations different approaches provide #2020-07-28 19:33:37 alexmiller when you have static data, the best thing to do is put it in deps.edn and reference via alias #2020-07-28 19:34:30 seancorfield > including things that can be read as collections -> string
I think that's a bad idea since it's common to want to pass a map of stuff when calling Clojure function (in general) so that choice would force EDN parsing on everyone #2020-07-28 19:35:02 alexmiller with -X you are always passing a map - the command line is overrides to the data in deps.edn #2020-07-28 19:35:18 alexmiller and the overrides support nested paths #2020-07-28 19:35:20 seancorfield Right, I meant as one of the arguments (inside that map) #2020-07-28 19:35:40 alexmiller nested paths gets you out of another set of cases #2020-07-28 19:35:45 seancorfield i.e., to produce a nested map #2020-07-28 19:36:13 vlaaad like -X:deploy :envs '#{"prod" "qa"}' #2020-07-28 19:36:26 seancorfield Yeah, having :test 1234 and :test [1234 5678] produce a number and a string respectively would be very strange. #2020-07-28 19:36:38 vlaaad I agree 🙂 #2020-07-28 19:36:44 alexmiller it turns out vectors are actually ok bash if you use the comma #2020-07-28 19:36:55 vlaaad not ok pwsh #2020-07-28 19:37:04 alexmiller :test [1234,5678] is actually ok #2020-07-28 19:38:42 seancorfield I sympathize with the additional pain on PS (I too use it from time to time with the Clojure CLI) but since the vast majority of Clojure CLI users are on macOS/Linux I'd rather not make all their lives more painful, just to make PS less painful -- and it's already a bit painful to use for the Clojure CLI... #2020-07-28 19:40:55 alexmiller maybe this is a topic for #clj-on-windows but what's the story on the new Windows terminal? #2020-07-28 19:40:56 vlaaad hmm, I might be tripping, [1,2,3] worked fine in pwsh #2020-07-28 19:48:15 seancorfield and clj -X:example [:test,:value] 1234 works on macOS so, yay, portability! 🙂 #2020-07-28 19:45:20 seancorfield @alexmiller The new Windows terminal is just a fancy tabbed shell around existing PS, cmd, and Linux terminals. #2020-07-28 19:45:52 alexmiller so nothing new #2020-07-28 19:47:05 nate out of curiosity, why are unqualified lib names now deprecated? was there a deeper issue that needed addressing or is it just because qualified names are unambiguous and more correct? #2020-07-28 19:48:11 nate ah #2020-07-28 19:48:47 nate which mentioned the deprecation again, but I was curious if there was a deeper reason for it #2020-07-28 20:25:31 dominicm To encourage use of groupid #2020-07-28 20:26:21 dominicm And discourage hiccup/hiccup in favor of com.weavejester/hiccup. But in a general sense, not the global sense. #2020-07-28 20:29:20 sveri Finally 🙂 That was one thing that always made me wonder why one would have unqualified library names when clojure is so heavily rooted in java. #2020-07-28 19:52:15 alexmiller not sure what I can add beyond what's in the blog #2020-07-28 19:52:58 alexmiller unqualified names are bad so we're going to stop encouraging them in this area #2020-07-28 19:53:50 nate that makes sense, thank you #2020-07-28 20:17:23 vlaaad PS C:\Users\Vlaaad\Projects\tdeps> cat .\deps.edn
{:aliases {:prn {:fn clojure.core/prn}}}
PS C:\Users\Vlaaad\Projects\tdeps> clj -Sdescribe
{:version "1.10.1.596"
..."}
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/tools/deps/alpha/exec__init.class, clojure/tools/deps/alpha/exec.clj or clojure/tools/deps/alpha/exec.cljc on classpath. #2020-07-28 20:21:00 alexmiller hrm, let me check on that, shouldn't be using that class anymore #2020-07-28 20:21:53 vlaaad -Spath is
C:\Users\Vlaaad\.m2\repository\org\clojure\clojure\1.10.1\clojure-1.10.1.jar;C:\Users\Vlaaad\.m2\repository\org\clojure\spec.alpha\0.2.176\spec.alpha-0.2.176.jar;C:\Users\Vlaaad\.m2\repository\org\clojure\core.specs.alpha\0.2.44\core.specs.alpha-0.2.44.jar;src
` #2020-07-28 20:22:49 alexmiller path is fine, it's the class, I missed one change there #2020-07-28 20:25:05 alexmiller windows 1.10.1.600 has the fix #2020-07-28 20:26:43 dominicm Does -X compose? #2020-07-28 20:28:24 borkdude What do you mean by that? #2020-07-28 20:28:32 alexmiller depends what you want to compose with :) #2020-07-28 20:28:36 alexmiller with other -X, no #2020-07-28 20:28:43 alexmiller with all the classpath stuff, yes #2020-07-28 20:29:04 alexmiller but if you want to run multiple things, write a program that does so and run it with -X :) #2020-07-28 20:31:05 dominicm I don't want to write lots of programs! I want to change my programs on a whim at the cli. Start a socket server, start a whatever, run the program in the local project that does setup, run my custom dev function. #2020-07-28 20:31:50 dominicm (We've discussed before though, not really trying to convince, but I hoped that exec would provide this) #2020-07-28 20:32:49 dominicm Right now I'm (ab)using the fact you can supply an init and a main to load a dev.clj in my home directory, which will run things based on the directory the jvm started in... #2020-07-28 20:34:18 alexmiller there's this thing called a repl that lets you do like ... anything #2020-07-28 20:35:09 dominicm Yeah, but I have to like type all these weird ( characters. #2020-07-28 20:35:39 vlaaad PS C:\Users\Vlaaad\Projects\tdeps> clj -Sdescribe
{:version "1.10.1.600" ...}
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
Execution error (ExceptionInfo) at clj-exec/check-first (clj_exec.clj:32).
Invalid first arg to exec: prn
#2020-07-28 20:36:33 vlaaad could it be PS escaping away the :? 🤔 other args like -A:alias work fine thogh #2020-07-28 20:47:03 alexmiller will take a look, not sure #2020-07-28 20:50:33 alexmiller maybe I'm missing the first arg I match in the pwsh for this, but seems like we'd get nothing here, not part of the arg #2020-07-28 20:53:09 alexmiller how hard is it for you to just modify the ClojureTools.psm1 ? #2020-07-28 20:54:08 alexmiller I think line 84 just needs to be $ExecAlias += $arg, $params #2020-07-28 20:54:28 alexmiller it's losing the matching $arg right now #2020-07-29 18:32:53 vlaaad I tried what you suggested, now clj "-X:prn" :a 1 succeeds, while clj -X:prn :a 1 still fails with error Invalid token: : #2020-07-29 18:42:02 alexmiller hrm #2020-07-29 18:48:52 vlaaad hacked it to work with this:
$sym, $params = $params
$ExecAlias += "$arg$sym", $params
now it doesn't work with "-X:prn" though #2020-07-29 18:51:05 vlaaad seems to be PS parsing issue:
> echo -X:prn
-X:
prn
#2020-07-29 18:51:47 vlaaad > echo -A:prn
-A:
prn
#2020-07-29 18:52:28 vlaaad hmm, that works:
s> clj -A: beep:boop:bap
WARNING: Specified aliases are undeclared: [:beep :boop :bap]
Clojure 1.10.1
user=>
#2020-07-29 18:54:10 alexmiller is there a difference in powershell parsing a single thing (into leading char / remaining char) or many things? #2020-07-29 18:55:16 vlaaad you mean -XXX:yyy vs -X:y ? #2020-07-29 18:58:49 alexmiller no, I mean are we getting different behavior on clj -X:foo vs clj -X:foo :a 1 ? #2020-07-29 19:01:34 vlaaad no:
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn :a 1
{:a 1}
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
nil #2020-07-29 19:02:47 vlaaad but this wont work: clj "-X:prn", because I made the script to expect args -X: and prn separately #2020-07-29 19:06:24 vlaaad so what we really need to do is accept both -X:alias and -X: , alias for powershell to make it predictable #2020-07-29 19:06:37 vlaaad I'll try to write some powershell.. #2020-07-29 19:11:58 vlaaad if ($arg -eq "-X:") {
$sym, $params = $params
$ExecAlias += "$arg$sym", $params
} else {
$ExecAlias += $arg, $params
} #2020-07-29 19:12:16 vlaaad PS C:\Users\Vlaaad\Projects\tdeps> clj "-X:prn" :a 1
{:a 1}
PS C:\Users\Vlaaad\Projects\tdeps> clj "-X:prn"
nil
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn
nil
PS C:\Users\Vlaaad\Projects\tdeps> clj -X:prn :a 1
{:a 1} #2020-07-29 19:24:29 alexmiller the code for other aliases in tools.deps is actually tolerant of the split here, maybe that's necessary for the -X stuff too. in other words, we've been masking this problem #2020-07-28 20:37:28 dominicm In earnest, there's a little bit of a desire to hide implementation behind flags. Telling someone to run clojure -A:nrepl:piggieback is easier than explaining that they need to run a set of functions once waiting for the repl to connect, and then press buttons on their editor, in an exact sequence to make it work.
Right now I use jvm options. #2020-07-28 20:56:13 seancorfield Dang, we use a lot of single-segment lib names at work! 😞 #2020-07-28 20:57:39 seancorfield (updating 27 deps.edn files in our monorepo!) #2020-07-28 21:00:27 alexmiller feedback on the warning message? I tried to make it useful #2020-07-28 21:03:38 seancorfield It's perfect. #2020-07-28 21:03:42 seancorfield Annoying, but perfect. #2020-07-28 21:04:01 seancorfield The only slightly weird thing was that I seemed to get all of the warnings twice? #2020-07-28 21:12:33 alexmiller that dimly rings a bell. I wrote this like a month ago. I think because it's encountered during canonicalization and also during expansion or something? #2020-07-28 21:00:49 seancorfield Am I right in thinking the new "alias data" feature is going to allow people to add all sorts of configuration stuff into deps.edn as data under an alias? #2020-07-28 21:01:08 alexmiller if they like #2020-07-28 21:01:17 seancorfield Folks have been asking for that for quite a while 🙂 #2020-07-28 21:01:26 alexmiller like most things, taste is helpful #2020-07-28 21:01:36 seancorfield (and have been told in the past, "don't do that" 🙂 ) #2020-07-28 21:01:44 alexmiller it might be a good place to put like, build configuration #2020-07-28 21:02:34 seancorfield Hmm, and because aliases don't compose, it's "just" fixed data so I guess that's a nice place to draw the line. #2020-07-28 21:03:21 seancorfield I suspect shadow-cljs and other tools might leverage it for build config to avoid having a separate file for that... #2020-07-28 21:07:16 dominicm Aliases can compose, but it's down to their interpreter to decide how. #2020-07-28 21:08:39 alexmiller btw, I don't know if add-libs works with latest (haven't tried). I am still injecting the lib map atm which is what add-libs works starts from, but not sure if it also uses anything else in tools.deps that's changed. the branch I have has the stuff to switch to basis but also other stuff too so some interpolation will need to be done, not sure when I'll get to that #2020-07-28 21:09:44 seancorfield NP. I just updated my dot-clojure deps.edn to rename the alias from :deps to :add-lib and add notes that it should be considered experimental and likely to break or go away 🙂 #2020-07-28 21:10:14 seancorfield (I had forgotten that the installed deps.edn contains a :deps alias when I originally added it) #2020-07-28 21:11:09 seancorfield I also just cleaned up the :rebl* aliases to reflect the latest REBL having its own deps.edn files for java8 and openjfx15ea (plus some reorganization and updating several versions). #2020-07-28 21:11:40 alexmiller cool #2020-07-28 21:13:57 borkdude My work is now finally using deps.edn/tools.deps/clojure CLI (how do you even call this stuff?). I moved out our front-end build to it (using figwheel.main / cljs.main) last week while we still do the uberjar and REPL stuff with boot. Also now we can finally run Java 11 everywhere, since we were stuck, the boot frontend stuff didn't work with it (at least not for us). #2020-07-28 21:15:06 ghadi that's exciting @borkdude #2020-07-28 21:15:37 alexmiller nice! #2020-07-28 21:20:47 dominicm 3 jvms! I bet that doesn't integrate well with cider jack in. Might I suggest starting 3 threads instead? #2020-07-28 21:21:36 borkdude I've always avoided cider-jack-in, I always use cider-connect. Even more so now I'm developing on a remote machine but editing files in my local emacs using tramp. #2020-07-28 21:25:40 borkdude Threads may work but it's not a huge deal right now. We need at least two JVMs given we're using two different build/classpath/dependency tools right now. #2020-07-28 21:26:14 seancorfield "I've always avoided cider" -- ftfy 🙂 #2020-07-28 21:27:46 seancorfield As part of this monorepo deps cleanup I'm doing (to prevent all those DEPRECATED warnings!) I'm also removing the last few references to nREPL in our setup since neither I nor my team mate use that now. #2020-07-28 21:28:04 dominicm seems unfair to people who use jack in. ¯\(ツ)/¯ #2020-07-28 21:28:25 borkdude Well, I asked my teammates and this was all OK for them, so I guess we're good 🙂 #2020-07-28 22:36:59 rickmoynihan I’m curious why the new tools.deps adds a tool to do local maven installs. Is this something people frequently need?
I’ve probably only ever needed to install free form jars into my local maven repo a few times in the past ten years. I mean sure if I’ve built a jar myself I will frequently want to install it, but it seems strange to supply a method to install jars, but not build them.
I’m just curious what problem it’s solving for people. #2020-07-28 22:39:06 rickmoynihan Am I missing something in my workflow; or is it just that this happens to have been baked before other build tasks in tools.build? #2020-07-28 22:56:37 kenny Perhaps related to Datomic's dev-local needing to be installed into your local Maven repo to be used (it's not distributable). #2020-07-28 23:03:38 seancorfield ^ That is what Alex said when asked earlier @rickmoynihan #2020-07-28 23:06:41 rickmoynihan ahh ok makes sense in that use case. Thanks @seancorfield #2020-07-29 01:15:01 seancorfield (OK, running 1.10.1.596 on all our servers at work now 🙂 ) #2020-07-29 01:50:42 salam how would one represent this in deps.edn?
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc-bom</artifactId>
<version>19.7.0.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc10</artifactId>
</dependency>
</dependencies> #2020-07-29 01:51:37 salam if possible, of course. #2020-07-29 02:03:42 seancorfield @abdusalam Pretty sure you'll need to figure out what that Bill Of Materials expands to and add each of those dependencies directly to deps.edn #2020-07-29 02:06:25 salam thanks, Sean. does it mean there's no direct, built-in support in tools.deps for this kind of use case? #2020-07-29 02:25:55 seancorfield Not as far as I'm aware. #2020-07-29 02:35:27 alexmiller no, deps does not support the BOM / import scope stuff #2020-07-29 02:38:33 alexmiller it seems radical, but deps just expects you to list the specific dependencies you're using :) #2020-07-29 03:03:00 salam i see. oracle's jdbc driver (com.oracle.database.jdbc/ojdbc10) used to list other "extra goodies" as dependencies in its pom.xml. i was looking for an elegant way to exclude the extra stuff in order to avoid doing the "exclusion dance" but just found out that it's dropped all of the dependencies since version 19.7.0.0. all we need to do now is to add com.oracle.database.jdbc/ojdbc10 {:mvn/version "19.7.0.0"} to get just the driver itself. 🙂 #2020-07-29 03:03:49 salam all good now #2020-07-29 03:23:20 alexmiller yeah, that's all most people need. the whole bom thing seems like something an enterprise java developer thought up while they were high #2020-07-29 14:05:30 alexmiller the docs are (always) a work in progress. I pulled the diagram b/c it was way out of date. I have updated it a number of times during the process of the basis changes but I have concerns about whether it actually provides that much value as it has grown increasingly complicated. All of the info from the old Aliases section is still in the doc. #2020-07-29 19:50:14 onetom i found that diagram quite helpful, when i was initially learning the clojure cli.
in practice however, i never really had to use anything else then -A so far... #2020-07-29 07:02:02 rickmoynihan Second question — how stable are the new features in tools.deps? In particular the -X feature and the args map data? Are they likely to experience breaking changes between now and the next stable release? I get the feeling not.
Sometimes it’s not clear to me what is blessed as stable and isn’t… i.e. the installer, the command line args, or the tools.deps api. (Though I guess it’s all technically alpha because it’s t.d.a not t.d) #2020-07-29 13:09:08 alexmiller should be stable unless we discover some reason to change. not expecting to discover anything but that's why we're letting people try it. I'm expecting the gap to stable to be like days, not a long time. #2020-07-29 15:55:54 seancorfield I was surprised that 1.10.1.590 seems to be the latest stable version on brew already, given the flurry of bug fix releases yesterday (1.10.1.596 and 1.10.1.600). #2020-07-29 16:40:39 alexmiller It shouldn’t be? #2020-07-29 16:43:25 alexmiller So I’m curious why you said that #2020-07-29 16:43:33 seancorfield I just did brew upgrade clojure on WSL1 on my Windows laptop last night and it updated me to 590. Hence my surprise. #2020-07-29 16:43:42 seancorfield Note: brew on Linux. #2020-07-29 16:43:53 alexmiller That’s homebrew core, which I no longer update #2020-07-29 16:44:38 alexmiller clojure/tools/clojure is official #2020-07-29 16:44:53 seancorfield I know. Hence my surprise 🙂 #2020-07-29 16:48:12 alexmiller Well I released 590 a week ago, just took me a while to doc and announce #2020-07-29 16:48:24 alexmiller But I guess somebody updated it #2020-07-29 10:46:17 thheller @alexmiller how come you are dumping all the new features into :aliases? kinda confusing to give aliases multiple meanings IMHO. maybe I'm just not seeing the bigger picture though. #2020-07-29 10:53:23 thheller like why is the exec stuff in :aliases and not a new top-level :exec key or so? #2020-07-29 11:05:46 borkdude as far as I have understood it: :aliases was always intended for naming a chunk of EDN data that could be re-used elsewhere, but it's only now that we can see that being used in more places #2020-07-29 13:07:37 alexmiller correct - I would flip it around to say :aliases is a generic facility, and we are making more use of it #2020-07-29 11:06:54 borkdude I wonder if you can combine -A and -X to include extra deps in an -X invocation #2020-07-29 13:06:32 alexmiller you can, absolutely #2020-07-29 13:06:44 alexmiller that's kind of the idea #2020-07-29 13:51:23 rickmoynihan 🤔 so you can change the function that executed with the default data (by changing the classpath via :extra-deps); but you can’t change the data for a function alias #2020-07-29 14:00:51 alexmiller I'm not sure that's "changing the function" as much as supplying the function #2020-07-29 14:00:58 alexmiller and you can override the data on the command line #2020-07-29 14:01:07 alexmiller maybe I'm not getting what you're describing #2020-07-29 14:02:04 rickmoynihan is that intended usage?
It feels a bit weird to have the same fully qualified function name, but with two different definitions (in two different :paths for example).
Would it not be more natural for -X :args to compose? #2020-07-29 14:02:25 alexmiller I don't understand why you have that #2020-07-29 14:02:52 alexmiller can you more fully describe what you're talking about, because I don't get it #2020-07-29 14:10:30 vlaaad @rickmoynihan you mean something like that?
{:aliases {:deploy-to {:fn my.ns/deploy}}
:prod {:args {:url ""}}
:qa {:args {:url ""}}}
clj -X:deploy-to:prod #2020-07-29 14:14:56 rickmoynihan yes #2020-07-29 14:26:34 vlaaad I thought it would work, but it doesn’t.. #2020-07-29 14:37:14 alexmiller yeah, that's not a thing #2020-07-29 14:40:00 alexmiller I would probably do this:
{:aliases
{:deploy-prod
{:fn my.ns/deploy
:args {:url ""}}
:deploy-qa
{:fn my.ns/deploy
:args {:url ""}}} #2020-07-29 14:40:20 alexmiller but there may be other variants depending on shared config, command-line overrides etc #2020-07-29 14:40:55 alexmiller also my.ns/deploy could interpret data in :args as something to look up in :aliases via the basis #2020-07-29 14:41:14 alexmiller you don't get that for free, but it's like 2 lines of code #2020-07-29 14:55:22 vlaaad Yeah, I get what you mean. Basis FTW! #2020-07-29 15:59:17 seancorfield The basis stuff looks very interesting, but you lose access to it if your build your artifacts as uberjars and run them via java -jar (although I guess you could still do -Dclojure.basis='{...}' 👀 ) #2020-07-29 16:00:00 seancorfield It almost makes me want to go back to running code in production via clojure instead 🙂 #2020-07-29 14:11:27 rickmoynihan Yeah sorry… I think I misinterpreted the motivation behind what @borkdude was saying as a way of effectively doing what @dominicm was asking earlier about composing :args data.
But I misread :extra-deps as being :deps.
Either way it seems :deps would kind of allow the kind of thing @dominicm was asking for; but in a nasty way… i.e. you could swap the function rather than the data. e.g.
- clj -A:myfn -X:args
- clj -A:myfn2 -X:args
Where myfn and myfn2 aliases defined the same ns but with different function definitions. #2020-07-29 14:18:54 dominicm I want to run fn2 and fn1 in this example. Tbh, I assumed args and fn had to co exist. #2020-07-29 14:20:44 alexmiller I'm unclear if you have a question, and if so, what it is... #2020-07-29 14:21:10 dominicm Not a question, I already asked mine! #2020-07-29 14:21:25 dominicm Just clarifying that I don't think this would allow for multiple functions like on after :) #2020-07-29 14:24:45 alexmiller no #2020-07-29 14:11:34 rickmoynihan Not saying this is a good idea btw!!! #2020-07-29 14:12:06 alexmiller one helper here is that :args can also refer to an alias instead of a map #2020-07-29 14:13:58 vlaaad that’s something new! #2020-07-29 14:12:22 rickmoynihan oh right… that is interesting! #2020-07-29 14:12:46 rickmoynihan seems more useful than the :paths one #2020-07-29 14:13:34 alexmiller well the :paths one is foreshadowing :) #2020-07-29 14:14:14 alexmiller I did forget to mention that in my blog writeup, but I think it's in the docs #2020-07-29 14:14:57 alexmiller or maybe it isn't! #2020-07-29 14:15:19 rickmoynihan I don’t recall seeing it #2020-07-29 14:15:28 vlaaad yeah, me neither #2020-07-29 14:15:38 alexmiller it was in some version of the docs but looks like it got lost in the N reworks I did #2020-07-29 14:15:45 rickmoynihan 😆 easily done #2020-07-29 14:15:47 alexmiller anyhow, that's a thing #2020-07-29 14:16:23 rickmoynihan wow — I was silently disappointed that something like this wasn’t there #2020-07-29 14:16:30 rickmoynihan but it’s great that it is #2020-07-29 14:16:36 rickmoynihan is there an example of how it works? #2020-07-29 14:23:44 alexmiller {:aliases
{:f1 {:fn my/fn1
:args :data}
:f2 {:fn my/fn2
:args :data}
:data {:something :here}}}
and then
clj -X:f1 or clj -X:f2 #2020-07-29 14:17:31 vlaaad where else can I put aliases?!?!?! #2020-07-29 14:17:43 alexmiller data #2020-07-29 14:17:51 alexmiller it's just edn #2020-07-29 14:18:01 alexmiller oh sorry, misread your question #2020-07-29 14:27:41 dominicm Now we have runtime basis, does that mean you can go back and interpret the exec aliases however you like? #2020-07-29 14:36:11 alexmiller you can write a program that uses alias data #2020-07-29 14:36:30 alexmiller which is broader than what you said #2020-07-29 16:03:06 seancorfield I asked in a thread, but I'll ask in the main channel for everyone to see: given that clojure.basis is a property added by the clojure command, what is the thinking behind leveraging this in production code if you normally build uberjars and run those with java -jar? All of that lovely combined deps.edn structure has gone by that point... #2020-07-30 09:11:14 rickmoynihan I also had a similar thought when I stumbled across the basis property etc.
Though I figured you could spit out the basis to a clojure.basis file inside the uberjar, and slurp it as an io/resource in your -main.
However the big disadvantage to this is that the basis would be generated at build time rather than run time, so it would be more like a manifest, than what the basis is really supposed to be.
On reflection I think your idea is better — though I’m not entirely sure what you’d use the basis for in that case, other than as a means of using the clojure tool to provide config to your app. #2020-07-30 09:21:22 rickmoynihan I guess the only real advantage is using the clojure tool and a deps.edn to manage the classpath in production code, rather than the underlying java command line. #2020-07-30 09:25:40 rickmoynihan I have in the past done things like java -cp:myapp-uberjar.jar:some/server/resources/* myapp.main
to for example add assets to an apps resource path as a production overlay… so I guess using the clojure tool itself to help manage these kind of things with aliases might be useful; though I’m not convinced the benefits are huge #2020-07-30 09:26:32 rickmoynihan though I guess it might sometimes be useful to for example provide profiles in production for enabling socket servers etc 🤔 #2020-07-30 09:30:38 rickmoynihan Also another usecase is you could provide extra tooling for prod systems in this manner.
e.g. a set of aliases for various production tasks, e.g. triggering a -A:backup alias over cron etc. #2020-07-30 09:31:42 rickmoynihan and it would mean you don’t need to waste time inventing new command line parsers and config formats etc #2020-07-29 16:04:13 seancorfield It almost feels like it's worth switching to using clojure to run uberjars via an alias in the deps.edn so that all that stuff remains available in a "production" style context... #2020-07-29 16:04:24 ghadi my guess is that an uberjar tool could convey the basis into the jar #2020-07-29 16:05:17 ghadi the more I think about uberjars, the more disadvantages I see vs. sync'ing deps #2020-07-29 16:05:34 seancorfield The basis differs depending on the aliases used to run clojure tho', right? #2020-07-29 16:06:32 seancorfield (because the paths and lib map etc can be different from invocation to invocation -- although the set of aliases etc remains the same) #2020-07-29 16:06:44 ghadi I'm shooting from the hip #2020-07-29 16:07:35 ghadi i'd mostly want the alias data #2020-07-29 16:07:42 ghadi for stuff like config #2020-07-29 16:08:38 seancorfield I'd add it to depstar but I think at this point I want to wait and see what tools.build ends up being 🙂 #2020-07-29 16:11:08 seancorfield I might experiment with using clojure to run uberjars tho' later today puts on mad scientist hat... #2020-07-29 16:11:22 ghadi wdym? #2020-07-29 16:12:27 ghadi using clojure instead of uberjars? #2020-07-29 16:12:31 seancorfield Having an alias that treats the uberjar as a :local/root dependency and runs the expected -main function inside it, so you get easy access to the t.d.a. / Clojure CLI infrastructure #2020-07-29 16:19:43 seancorfield This:
:aliases
{:run {:deps {worldsingles/worldsingles {:local/root "../../build/uberjars/worldsingles-1.0.0.jar"}}
:fn worldsingles.publisher/-main}}
...
And then clojure -X:run #2020-07-29 16:20:38 seancorfield So you get the "benefit" of an AOT'd uberjar with the benefits of the Clojure CLI. And the program can read the basis 🙂 #2020-07-29 16:23:51 seancorfield (of course, now you need the Clojure CLI installed on a target server, not just the JVM, and you need a deps.edn file in the folder where you start the app) #2020-07-29 16:35:15 vlaaad And internet access #2020-07-29 16:35:25 vlaaad And credentials for private repos #2020-07-29 16:37:32 seancorfield @vlaaad Not if the entire :deps is just the uberjar, right? #2020-07-29 16:37:55 seancorfield There are no other deps to fetch at that point. #2020-07-29 16:40:39 vlaaad Ah, sorry, misread the situation #2020-07-29 16:41:05 vlaaad I was thinking about using clj without uberjar #2020-07-29 16:42:41 seancorfield Right, which we used to do on production (and, on one server, for "reasons" we still do that). #2020-07-29 18:31:49 borkdude Not that I really care, just wanted to report that if you call a private function with -X, it also works:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "61030e130a05140504212c233153515058"}, :content ("[email protected]")} #2020-07-29 18:44:06 alexmiller that's not intentional but interesting :) #2020-07-29 18:44:21 alexmiller it does make sense given the impl #2020-07-29 18:46:11 seancorfield Because requiring-resolve can get at private Vars and there's no access check on that dynamic call path, right? #2020-07-29 18:48:56 alexmiller right, just like calling through a var #2020-07-31 21:09:31 borkdude The private var call still works with 619 🙂 #2020-07-31 21:16:09 alexmiller I'm ok with that #2020-07-31 21:16:22 alexmiller I don't consider it a bug #2020-07-31 21:16:37 alexmiller private functions are only in your mind #2020-07-31 21:17:26 borkdude I'm going to quote you on that when somebody will ask questions about this 😂
$ bb '(ns foo) (defn- foo [] :foo!) (ns bar (:require foo)) (foo/foo)'
:foo!
#2020-07-29 18:40:40 p14n ANybody else having issues with the recent tools-deps docker? #2020-07-29 18:42:12 p14n Specifically
Caused by: Syntax error compiling . at (aleph/http/core.clj:251:3).
Caused by: java.lang.IllegalArgumentException: No matching method setTransferEncodingChunked found taking 2 args for class io.netty.handler.codec.http.HttpUtil
#2020-07-29 18:42:35 p14n (I'm making assumptions here because nothing else has changed) #2020-07-29 18:44:39 seancorfield @p14n change in JDK version perhaps? Last night I was helping some beginners debugging a weird reflection-based problem that changed between JDK 11 and JDK 14 (it was broken on 8 and 11 but worked on 14). #2020-07-29 18:46:24 p14n That's what I thought. I switched to openjdk-8-tools-deps (tools-deps now 11) but no dice yet #2020-07-29 19:12:48 p14n Fixed by using clojure:openjdk-8-tools-deps-1.10.1.502 :shrug: #2020-07-29 19:13:32 seancorfield A change in the default Clojure version perhaps, with stricter syntax checking? #2020-07-29 19:14:52 ghadi i don't use any of the clojure:* images #2020-07-29 19:15:09 ghadi they are not made by clojure team, and are not necessary IMHO #2020-07-29 19:15:46 ghadi I use an adoptopenjdk image and then add the clojure installer #2020-07-29 19:19:12 p14n Do you host your own? #2020-07-29 19:15:27 cap10morgan How is the groupId derived in clojure -Spom? mine keeps getting reset to the artifactId when I run it. #2020-07-29 19:16:33 seancorfield That's a bug. That should not happen. #2020-07-29 19:16:42 seancorfield (and it's a new bug ^ @alexmiller) #2020-07-29 19:17:27 cap10morgan ah, interesting. this is with 1.10.1.590. #2020-07-29 19:17:33 alexmiller I don't think anything has changed there #2020-07-29 19:17:49 seancorfield (! 1133)-> head pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi="" xsi:schemaLocation=" ">
<modelVersion>4.0.0</modelVersion>
<groupId>seancorfield</groupId>
<artifactId>next.jdbc</artifactId>
<version>1.1.569</version>
<name>next.jdbc</name>
<description>The next generation of clojure.java.jdbc: a new low-level Clojure wrapper for JDBC-based access to databases.</description>
<url></url>
<licenses>
Wed Jul 29 12:17:19
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1134)-> clojure -Spom
Wed Jul 29 12:17:30
(sean)-(jobs:0)-(/Developer/workspace/next.jdbc)
(! 1135)-> git diff
diff --git a/pom.xml b/pom.xml
index 3dce875..5632254 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi="" xsi:schemaLocation=" ">
<modelVersion>4.0.0</modelVersion>
- <groupId>seancorfield</groupId>
+ <groupId>next.jdbc</groupId>
<artifactId>next.jdbc</artifactId>
<version>1.1.569</version>
<name>next.jdbc</name>
#2020-07-29 19:18:14 seancorfield The only change it makes is to overwrite the groupId #2020-07-29 19:17:58 alexmiller well, nothing intentional :) #2020-07-29 19:21:27 alexmiller can you file a jira, I'm juggling many conversations right now #2020-07-29 19:22:44 cap10morgan Will do #2020-07-29 19:25:30 cap10morgan @seancorfield The expected behavior was just that it should leave that value alone when the pom.xml already exists? #2020-07-29 19:31:02 souenzzo The function that turns (merge-cli-ops {:default :opts} "[:extra :opts]" "true") => {:extra {:opts true} :default :opts} is public? #2020-07-29 19:42:45 alexmiller sorry, I don't understand the question #2020-07-29 19:46:09 alexmiller or how it relates to tools.deps #2020-07-29 19:51:19 souenzzo I liked the "DSL" to "assoc-in" values. #2020-07-29 20:47:02 alexmiller Oh, yes that’s it. Treat anything about this code as implementation detail though #2020-07-29 20:47:27 alexmiller It may migrate into Clojure itself #2020-07-29 19:39:03 seancorfield @cap10morgan Yup, it should only touch the dependencies in an existing pom.xml file. #2020-07-29 19:43:20 alexmiller well, it's broader than that - deps, repos, directories #2020-07-29 19:44:00 alexmiller there has been some work done internally here to make this more configurable and that will eventually make its way into user configurability #2020-07-30 02:14:36 cfleming Is there anything in the new tools.deps stuff that would make removing the Clojure coming from the system deps file easier? My main use cases are using a fork of Clojure, and also using Deps to manage non-Clojure projects (i.e. parts of Cursive). #2020-07-30 02:16:23 seancorfield @cfleming Can't you just use :override-deps and an alias? #2020-07-30 02:16:42 cfleming I don’t think that works. #2020-07-30 02:17:17 cfleming I’ll play around with that and see, but IIRC it didn’t work last I tried. #2020-07-30 02:17:17 seancorfield As for non-Clojure stuff, nothing yet but maybe tools.build might fingers crossed ? #2020-07-30 02:17:39 cfleming tools.build? #2020-07-30 02:18:07 seancorfield I know you can select any released Clojure version via :override-deps so I assume you could point it to a forked artifact...? #2020-07-30 02:18:35 seancorfield tools.build was mentioned in one of Alex's blog posts with a "shhh, secret" comment and it's popped up occasionally here #2020-07-30 02:19:17 seancorfield I assume it's something Cognitect have developed internally, probably to help with Datomic (like the recent -X option in the Clojure CLI with the install option for Datomic dev-local). #2020-07-30 02:19:48 cfleming Maybe I will finally be able to get off Ant 🙂 #2020-07-30 02:21:57 cfleming @seancorfield Looking at the docs, :override-deps seems to only be about overriding versions, not changing the artifact. #2020-07-30 02:22:48 seancorfield I assumed your "artifact" would have the same group/artifact ID but the "version" would point to :local/root (a JAR)? #2020-07-30 02:27:27 cfleming Wow, what do you know - it works!
~/d/tools-deps-test> cat deps.edn
{:override-deps {org.clojure/clojure {:local/root "/Users/colin/.m2/repository/com/cursive-ide/clojure/1.10.1/clojure-1.10.1.jar"}}}⏎ ~/d/tools-deps-test> clj -Spath
/Users/colin/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:src #2020-07-30 02:28:53 cfleming Thanks @seancorfield, that may make my life much easier. #2020-07-30 02:29:30 cfleming For modules where I don’t want Clojure, I should be able to use that trick to point Clojure to an empty jar, and then just filter that out when building. #2020-07-30 02:42:26 seancorfield You just made my week! 🙂 #2020-07-30 02:43:17 cfleming And mine 🙂 #2020-07-30 02:44:34 cfleming Oops, wait a minute - I didn’t look closely enough. The classpath still refers to the original Clojure 😞 #2020-07-30 02:44:43 seancorfield Aw, sorry... #2020-07-30 02:46:28 cfleming Actually, no, the root cause is that I’m an idiot. Here’s the working version, with override-deps in an alias as it should be:
~/d/tools-deps-test> cat deps.edn
{:aliases {:fork {:override-deps {org.clojure/clojure {:local/root "/Users/colin/.m2/repository/com/cursive-ide/clojure/1.10.1/clojure-1.10.1.jar"}}}}}
~/d/tools-deps-test> clj -Spath -A:fork
/Users/colin/.m2/repository/com/cursive-ide/clojure/1.10.1/clojure-1.10.1.jar:/Users/colin/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/Users/colin/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:src #2020-07-30 02:47:33 seancorfield Oh, thank goodness!! #2020-07-30 16:36:45 dominicm One risk of the new runtime basis is that programs are hard-coded against them, e.g. a repeat of the lein-figwheel problem. So the clj tools become mandatory for tooling. Setting a system property is easier than whatever assumptions lein plugins make about the environment. We should perhaps dissuade ourselves from doing this 🙂 #2020-07-30 16:37:09 dominicm On another note, if I do store information inside of deps.edn (e.g. a figwheel build info), how could I reload it? Is that part of tools.build? #2020-07-30 16:41:06 hiredman the add-lib branch distinguishes between the initial basis (static set at launch) and the runtime basis mutable as you add libraries #2020-07-30 16:41:21 hiredman so I imagine you don't reload the basis #2020-07-30 16:41:43 hiredman you take it as an initial value and stick it in an atom and mutate that #2020-07-30 16:47:52 seancorfield There are definitely tradeoffs involved. When we got started with Clojure a decade ago, we used lein run in production and continued to use that approach when we switched to Boot (2015) and later to the Clojure CLI. It's only been since that switch that we started building uberjars instead I think, and for a long while, we did not AOT code for those. In the end, we added the AOT step purely to improve startup time (it made a big difference for a couple of our processes, so they could come back into the cluster faster after a deployment). We've always generally had our app configuration in external files -- separate from the build/run tooling -- and/or via system properties (or environment variables). But I don't have a concern about requiring our programs to be run via clojure if there are enough benefits to be worth switching from java -jar or java -cp. #2020-07-30 16:47:54 hiredman (add-lib2 I guess, add-lib still calls it the lib-map instead of the basis) #2020-07-30 16:49:33 seancorfield As for "clj tools become mandatory" -- we're sort of already in that place: to work with a given project pretty much requires you use whatever tooling that project has adopted, be it lein, boot, or clj. #2020-07-30 16:57:09 alexmiller the basis is just a jvm property that points to a file that is an edn map. right now, clj makes that and injects it but nothing there is "special". lein could make a basis map and pass it too. or a user could make it and set it in jvm opts in lein. #2020-07-30 16:59:01 dominicm > we're sort of already in that place:
That's true for projects, but not tools. To clarify, I mean that your project makes clojure cli mandatory (although it could support lein alongside clj too, some projects do this). I'm worried about tooling like lein-figwheel or lein-sass, etc. which is tied to a particular JVM start tool needlessly. #2020-07-30 16:59:06 alexmiller @dominicm rather than expecting to reload the basis, you could have the basis include a reference to a different file that you watch and reload (or whatever) #2020-07-30 17:00:43 alexmiller tooling that is tied to a particular tool (ie a lein plugin) was always not a good factoring of the problem. it is far better to write a tool that is independent and just code and then adapt it to tooling as needed (and to prefer tools that levy minimal requirements such that the adapter may be nothing) #2020-07-30 17:01:07 alexmiller some lein plugins are written well for this, some aren't #2020-07-30 17:02:09 dominicm The clojure tools have inherently encouraged a lot of this through encouraging adoption of clojure main, so there's an inherent reduction of coupling to only be Clojure itself for many tools. (Not mine, mine all read deps.edn files!) #2020-07-30 17:02:35 alexmiller I don't know enough about lein-figwheel or lein-sass or whatever to say on those (but having "lein" in the name seems like a flag) #2020-07-30 17:04:56 dominicm One consideration will be what is standard/non-standard in the runtime basis. Lein can't tell you what aliases were loaded, it's a semantic mismatch. It could tell you some information about the deps though. #2020-07-30 17:16:29 alexmiller docs the basis map return (more detail is needed, a bit of it is intentionally a little vague right now, but it's mostly pretty set). I have not yet updated the specs for tools.deps but I will get to it eventually #2020-07-30 17:17:33 dominicm To me, those seem like they should be namespaced to tools deps if you think it's OK for other tools to generate. With the exception of lib-map, which I could see being generated by other tools (boot/lein) #2020-07-30 17:29:19 seancorfield The overall map is the combined deps.edn hash map, with those four extra keys added, so you're already tied to the deps.edn structure in the clojure.basis property. Leiningen isn't going to be able to create that master-edn map anyway, unless it adopts t.d.a. and deps.edn as an option (alongside project.clj) and changes its dependency resolution approach. So it seems you're already tied to clojure.basis and those simple keys at this point -- no need to qualify them, IMO. #2020-07-30 17:30:41 seancorfield Perhaps Leiningen could/should produce a lein.basis file/property when running code, that reflects the hash map structure of the project.clj file? But I don't think you can expect portability between tools (and I don't think you should). #2020-07-30 17:33:11 seancorfield "Lein can't tell you what aliases were loaded, it's a semantic mismatch." -- clojure.basis doesn't tell you what aliases were used to run the program, only the resulting "basis" of those aliases, but it includes the whole merged master-edn so you get "all" aliases available -- and lein could provide the :aliases structure from project.clj (although it's a completely different beast). #2020-07-30 22:25:09 pmonks So interesting finding just now that might be useful to others. I operate a tiny little chat bot, that runs on a cheap / low-spec PAAS. Earlier today I switched from running it via the Clojure CLI (i.e. via something like clojure -m mybot.main) to constructing an uberjar and running that (i.e. via something like java -jar mybot.jar), mostly in order to reduce startup times (the PAAS doesn’t provide blue/green deployment at the subscription level I’m on, and the brief outage during scheduled restarts had been noticed and commented on by some users).
As expected, startup time basically went to zero. What I didn’t expect was how much of a memory saving there’d be too. Previously, my bot had stabilised at around 340MB total (heap & off-heap) memory usage, which was quite close to the 512MB total provided by the PAAS, and which has to cover all processes’ memory use (not just that of my bot’s JVM).
Since the switch to an uberjar, total memory usage has stabilised at around 210MB. While I didn’t confirm this in detail, my preliminary conclusion is that the extra ~130MB is allocated by the tools.deps machinery (after all that’s the only thing that changed), and is not released after tools.deps has done its thing and is no longer in use. #2020-07-30 22:37:34 alexmiller Those are separate jvms so I don’t think I buy that if that was your theory #2020-07-30 22:37:58 pmonks What is your theory? #2020-07-30 22:38:33 pmonks Note: the uberjar is not AOT compiled (except for the ns with the -main function). #2020-07-30 23:09:06 hiredman aot compilation is transitive, so unless you work to avoid it, aot compiling a single namespace will also aot all namespaces it depends on as well #2020-07-30 23:14:27 pmonks Is that true of (gen-class) ? #2020-07-30 23:15:06 pmonks That’s the only thing I’m using (since it’s required in order to generate an “executable JAR” that java -jar can run directly). #2020-07-30 23:15:40 pmonks That said, this is my first use of depstar, so perhaps it’s doing something like that… #2020-07-30 23:20:57 hiredman it is complicated, but if you are using gen-class in the ns form, then yes, everything is being aot compiled #2020-07-30 23:22:00 pmonks I think you’re onto something. I cracked open the uberjar and sure enough my code is in there twice in .clj and .class form. #2020-07-30 22:39:39 alexmiller I would buy it for a container around the clj call, not sure what the PAAS is or how it works #2020-07-30 22:40:07 pmonks Heroku dyno (“hobby” subscription level). It’s pretty simple - a VM, as best I can tell. I’m not using Docker or any other container-like deployment mechanism. #2020-07-30 22:40:29 alexmiller I guess I'm saying "is allocated by the `tools.deps` machinery" is not in your final process, but may consume resources in the container itself #2020-07-30 22:40:50 pmonks Does that JVM stick around though? Once the “app” JVM is started? #2020-07-30 22:40:54 alexmiller no #2020-07-30 22:41:40 alexmiller the script waits for it to exit #2020-07-30 22:41:49 alexmiller before running the app #2020-07-30 22:48:31 alexmiller the clojure script forks a child process to run the first jvm, waits for it to exit, then exec's the app which replaces the script with the final jvm process #2020-07-30 22:48:39 pmonks I’m out of ideas. I did look into whether the local “ephemeral” filesystem might be a RAM drive that counts towards the VM’s RAM total, but that’s not the case (the ephemeral filesystem is 6GB). #2020-07-30 22:49:08 pmonks (since my app’s dependencies total around 90MB - somewhat close to the 130MB delta) #2020-07-30 22:49:39 alexmiller I can easily envision a container expanding its memory footprint and thus taking more time in the first case or something like that #2020-07-30 22:50:38 alexmiller but certainly the time savings makes sense #2020-07-30 22:50:44 pmonks Right, but the dashboard I’m looking at shows total memory used within the VM (and doesn’t give me any idea of the total RAM of the physical hardware that VM is running on). #2020-07-30 22:51:27 pmonks Yeah, and startup time savings was the impetus for making this change. I’m just surprised (and wish I could explain) the additional savings in RAM usage. Especially given how substantial those savings were (memory use dropped by ~40%). #2020-07-30 22:55:19 pmonks Here’s what I see, fwiw, spanning the time of the deployment where I switched to an uberjar: #2020-07-30 22:56:05 pmonks That graph is basically flat, going back a month or two (to when I first deployed the bot). #2020-07-30 22:56:49 alexmiller well, that could be reserved memory based on the max used (affected by the first jvm) #2020-07-30 22:57:30 pmonks It isn’t though - that dashboard further breaks down the JVM’s heap and off-heap memory usage, and both of those have dropped by approximately the same %ages. #2020-07-30 22:57:46 alexmiller I don't really have any way to tell #2020-07-30 22:57:51 pmonks I wouldn’t expect Heroku to “keep counting” a JVM that’s been terminated, indefinitely. #2020-07-30 22:58:32 pmonks Let me see if I can repro locally… #2020-07-30 23:05:10 pmonks Different numbers obvs, but I think I see a similar pattern locally (using visualvm to look at just heap & metaspace usage, so definitely not an apples-to-apples comparison with Heroku). #2020-07-30 23:05:31 pmonks Here’s clojure -m mybot.main: #2020-07-30 23:05:59 pmonks And here’s java -jar mybot.jar: #2020-07-30 23:06:37 pmonks (look at “used”, specifically - the JVM has allocated different total amounts from the OS in these runs, which throws off the vertical scales between the two runs) #2020-07-30 23:08:59 pmonks For easier reading, here are the key numbers:
| clj | jar |
------------+--------+--------+
Heap | 83MB | 39MB |
Metaspace | 71MB | 60MB |
------------+--------+--------+
#2020-07-30 23:12:04 pmonks (similar pattern, albeit different numbers, on successive runs, and with a GC requested before taking measurements) #2020-07-30 23:13:20 pmonks No clue what’s going, tbh. I’m not one to look a gift horse in the mouth, but then unexpected changes of this magnitude also make me suspicious. 😉 #2020-07-30 23:15:27 alexmiller one difference is that the jvm memory maps the jars - and here you've got many jars vs 1 #2020-07-30 23:15:44 alexmiller might be some per-jar overhead you're avoiding? #2020-07-30 23:16:07 alexmiller another thing to try would be to use -Scp #2020-07-30 23:16:39 alexmiller clojure -Scp the.jar -m your.namespace #2020-07-30 23:17:01 alexmiller that would be still using the script, not computing a classpath but using 1 jar #2020-07-30 23:18:12 alexmiller guessing that's still the lower number #2020-07-30 23:18:57 alexmiller and the next test would be to get the script out of the way. grab the computed classpath with -Spath, then java -jar vs java -cp with that #2020-07-30 23:37:52 pmonks Yeah I was wondering about total # of open file handles, but it’s hard to come up with 130MB that way. 😉 #2020-07-30 23:38:16 pmonks @hiredman found something though - it looks like all my code is being AOT compiled (the uberjar contains both .clj and .class versions of all of my namespaces). I don’t know why that’s happening (this is my first time using depstar, so I may have messed something up there), though apparently (gen-class) (which is used on my “-main” ns) can cause transitive AOT compilation too. #2020-07-30 23:41:01 pmonks Would runtime compilation of Clojure generate a lot of long-lived memory usage like that though? That’s the bit I can’t quite wrap my head around - this extra memory isn’t being GCed. #2020-07-30 23:42:06 pmonks (as part of trying to keep memory usage down on this restricted runtime environment, the bot calls (System/gc) every hour - I know that doesn’t guarantee that the GC will kick in but I figure it can’t hurt, especially as this particular bot is 99% batch/offline functions so it doesn’t matter if it’s not highly responsive to user activity) #2020-07-30 23:49:35 hiredman have you set a max memory on the jvm? #2020-07-30 23:49:58 pmonks When running locally, yes: -Xmx300m (I keep it low to try to emulate the memory-limited Heroku environment). #2020-07-30 23:50:15 pmonks On Heroku it’s container managed: -XX:+UseContainerSupport #2020-07-30 23:50:41 pmonks (locally my JVM is 11.0.2+9, while Heroku is JVM 11.0.8, btw) #2020-07-31 00:23:35 ghadi (isn't container support the default now in jdk11?) #2020-07-31 00:31:21 seancorfield @pmonks "I don’t know why that’s happening (this is my first time using depstar, so I may have messed something up there)" -- no, that's just how AOT works. In order to compile your main namespace, it has to load (compile) all the namespaces it depends on so you get pretty much your whole program compiled to .class files. That's just normal. What (:gen-class) does is flag that particular namespace to be generated as a class with methods -- functions beginning with - (by default) will become named methods on that class, instead of (what Clojure usually does which is) to compile each function to its own class with an .invoke() method. So your -main function becomes a public static method of the class (which is what Java expects). #2020-07-31 00:33:08 pmonks I didn’t realise AOT-compiled-Clojure couldn’t “thunk” into on-demand-compiled-Clojure. #2020-07-31 00:33:22 seancorfield Leiningen tries to do some fancy post-AOT clean up I think, to delete a lot of .class files that aren't directly part of your project so you don't get "everything" compiled? Can't remember whether that is the default or an option. #2020-07-31 00:33:32 pmonks (I almost never use AOT, so have minimal experience with it) #2020-07-31 00:33:54 pmonks Yeah and all of my previous experience has been with leiningen. #2020-07-31 00:34:04 seancorfield If you don't want other nses compiled, don't :require them in your main ns -- instead use runtime require/`resolve` (or the nice new requiring-resolve in 1.10). #2020-07-31 00:34:28 pmonks Thanks - looking now… #2020-07-31 00:34:58 pmonks I don’t particularly care either way, tbh. Have just heard folks here recommend against AOTing much, if any, of a codebase (if possible). #2020-07-31 00:35:45 seancorfield One good test to run here would be to build your uberjar without -C and -m on depstar -- which would not AOT anything -- and then run it with java -cp path/to/the.jar clojure.main -m mybot.main #2020-07-31 00:36:40 seancorfield That will do compile-on-demand and if it behaves the same as the original clj memory usage, then compile-on-demand might be responsible for the larger memory usage. #2020-07-31 00:38:05 pmonks Ok now that my 5pm US-PT (actually midnight UTC) batch job is done, I’ll give that a try. 😉 #2020-07-31 00:42:22 pmonks So locally, removing AOT seems to have resumed the “higher memory usage” scenario. #2020-07-31 00:42:35 pmonks Next up, heroku env. #2020-07-31 00:45:11 seancorfield So it sounds like there's some overhead from loading source and compiling on demand that doesn't go away? I'm a bit surprised it doesn't all get GC'd eventually. #2020-07-31 00:47:07 pmonks Yeah me too. Though I only have up to 24 hours of continuous operation available to look at, since Heroku forcibly restarts the dyno (VM) at least once per day. (as mentioned above, that includes at least 24 calls to (System/gc)). #2020-07-31 00:49:34 pmonks Deployed. Now we wait for a bit for it to show up on the Heroku dashboard. #2020-07-31 00:51:47 seancorfield I can't say I noticed a difference at work when we switched from compile-on-demand to AOT, but then our smallest process runs with 1G heap so a few 10s of MBs wouldn't really be noticeable... #2020-07-31 00:53:24 pmonks Makes sense. I can definitely see the CPU cost of on-demand compilation at startup showing up. 😉 Memory graph still a bit inconclusive… #2020-07-31 00:54:06 hiredman Calling System/gc is silly. Predicting what effect (if any)calling that method has is hard, so calling it, and then trying to reason from the basis of calling it is not helpful #2020-07-31 00:54:42 pmonks Sure, but it can’t hurt in a non-interactive app. #2020-07-31 00:54:45 hiredman You should look at the gc logs, and look at the sizes of the generationd #2020-07-31 00:55:04 pmonks And even a multi-second pause in this bot is preferable to exceeding memory (which puts me in Heroku “R14” jail…). #2020-07-31 00:55:29 ghadi how will you be in jail if the memory is bounded? #2020-07-31 00:55:50 hiredman Calling System/gc will never stop you from running out of memory #2020-07-31 00:56:09 hiredman That is how automatic garbage collection works #2020-07-31 00:56:26 pmonks By exceeding the 512MB quota for everything inside the VM (JVM heap + JVM metaspace + memory required by any other processes + OS memory requirements (caches, etc.) + …). #2020-07-31 00:56:53 hiredman It can only monkey with the heuristics in the gc #2020-07-31 00:57:06 pmonks Regardless, I tested it first (obvs) and it improved memory utilisation over longer periods of time. #2020-07-31 00:57:11 hiredman You should set a lower max heap for the jvm then #2020-07-31 00:57:17 pmonks I don’t set the heap - the container does. #2020-07-31 00:57:22 pmonks Remember I’m running with -XX:+UseContainerSupport. #2020-07-31 00:57:33 hiredman Then have the container do a better job #2020-07-31 00:57:40 pmonks Not to mention that controlling the heap doesn’t help with metaspace anyway. #2020-07-31 00:57:41 hiredman If you are exceeding the limits #2020-07-31 00:57:46 pmonks (which is where on-demand compiled classes etc. go) #2020-07-31 00:58:04 pmonks Know anyone who works at Heroku to improve their VM impl.? Cause I don’t. 😉 #2020-07-31 00:58:17 pmonks It’s not a container. #2020-07-31 00:58:21 pmonks It’s a “dyno”. #2020-07-31 00:58:32 pmonks I don’t provide anything but the stuff that gets fed to the JVM. #2020-07-31 00:59:03 pmonks (and while Heroku also supports Docker, I have negative interest in switching to that model of app construction) #2020-07-31 00:59:55 pmonks So it’s still a bit early to be sure, but it looks like it was AOT that was saving much of the memory: #2020-07-31 01:00:12 pmonks (that second “hump” is when I deployed with AOT disabled) #2020-07-31 01:01:10 hiredman You should check the different generations, compilation may result in extra old generation garbage, that is never collected because it never needs to be #2020-07-31 01:03:24 pmonks From what I’m seeing most of the savings are in the metaspace, not the heap. #2020-07-31 01:03:36 pmonks (recalling that old generation is on-heap) #2020-07-31 01:05:08 pmonks Which makes sense if on-demand compiled Clojure code is somehow more off-heap-expensive (e.g. metaspace, code-cache) than AOT compiled Clojure code. #2020-07-31 01:05:18 seancorfield In your original numbers, you only showed 11MB difference in metaspace, but 44MB in heap. #2020-07-31 01:05:27 pmonks Yeah that’s local. #2020-07-31 01:05:33 ghadi if you can AOT, AOT #2020-07-31 01:06:06 seancorfield Aye, AOT as the last step of an application, prior to production deployment seems like a reasonable step to me. #2020-07-31 01:06:11 pmonks @ghadi I probably will, given the memory restrictions in this environment. That doesn’t help me understand why this is happening though. #2020-07-31 01:06:36 pmonks To date I’ve never used AOT, and if there is a substantial (~40% memory saving, for this one bot) saving to be had, that could be a discriminating factor in choosing AOT vs on-demand compilation for other apps in future. #2020-07-31 01:06:48 seancorfield You'll need VisualVM or something similar to see what's actually going on inside the heap/metaspace... #2020-07-31 01:07:30 pmonks @seancorfield no easy way to hook that up to Heroku, though it runs an agent that does break down on-heap vs off-heap memory usage for their dashboard. #2020-07-31 01:07:58 seancorfield Right, I meant locally, with a low heap size to vaguely mirror what's happening on Heroku. #2020-07-31 01:09:47 pmonks Yeah - I’m doing that, but the absolute numbers are pretty different than what I see on Heroku. Possibly because it’s hard to limit the JVM’s off-heap memory usage (-Xmx only affects the heap), so it’s happily slurping up the (copious, compared to Heroku) memory on my laptop. #2020-07-31 01:11:11 pmonks BTW, here’s a more complete view of heap vs off-heap usage reported by heroku. The two deployments (the first to AOT uberjar, then back to non-AOT uberjar) are pretty obvious: #2020-07-31 01:11:46 pmonks Stand by - I accidentally chopped off “total memory usage”. #2020-07-31 01:11:55 ghadi we're in the wrong channel, btw #2020-07-31 01:12:17 pmonks We are now, yes. Originally this looked like tools.deps issue. #2020-07-31 01:12:48 ghadi I concur with @hiredman’s suggestion to get better details about heap regions. #2020-07-31 01:13:24 ghadi there are several interesting metrics depending on the GC used (G1 by default on 11) #2020-07-31 01:13:28 pmonks Here’s a better picture: #2020-07-31 01:15:07 pmonks Even at this granularity, it’s pretty clear that the delta in the non-heap memory usage is greater than that on-heap. #2020-07-31 01:15:18 pmonks i.e. looking at the heap regions likely won’t help #2020-07-31 01:16:37 pmonks (and yes the default Heroku dashboard sucks - it’s deliberately setup to encourage one to pay for a better one) #2020-07-31 01:18:35 pmonks Sadly I have to step away from this now, but it looks like AOT is causing the difference, even if it’s unclear why. Thanks for indulging me on this! #2020-07-31 01:31:05 ghadi not a mystery to me. compiling is more work than not compiling #2020-07-31 02:06:07 pmonks More work == CPU load, sure. But that work, once complete, shouldn’t leave long-lived garbage hanging around. #2020-07-31 02:08:21 pmonks Well, unless the JVM does tricks for AOT-compiled classes that it can’t use for dynamically generated (non-disk-resident) classes (e.g. mmap’ing pre-compiled .class files rather than keeping them resident in memory at all times, unloading disk-resident .class files once they’ve been JITted, etc.). #2020-07-31 02:28:53 pmonks Or if the Clojure compiler has caches that don’t get primed if it’s not called upon to dynamically compile some Clojure code. #2020-07-31 20:57:50 alexmiller A new clj release candidate (what we were formerly calling "dev" releases) is now available - 1.10.1.619 (see https://github.com/clojure/homebrew-tools#version-archive-tool-releases for installation info)
• Fixes -Spom regression in overwriting groupId in existing pom.xml files
• Improvements in error handling for -X
• New: -F execution specifier to invoke an arbitrary function that takes a map at the command line: clj -Fclojure.core/pr :a 1 :b 2 => {:a 1 :b 2} #2020-07-31 21:03:57 borkdude Awesome, I already wondered why one had to specify a function in deps.edn instead of just being able to call any function #2020-07-31 21:05:21 seancorfield (! 1038)-> clojure -Sdeps '{:aliases {:foo {:fn clojure.core/prn :args "test"}}}' -X:foo
Invalid :args for exec, must be map or alias keyword: "test"
(! 1039)-> clojure -Sdeps '{:aliases {:foo {:fn clojure.core/prn :args :bar} :bar "test"}}' -X:foo
"test"
🙂 #2020-07-31 21:08:58 seancorfield Thanks for the quick fix to pom.xml groupId! #2020-07-31 21:13:15 borkdude Why not just pass the entire map as one cmd line arg?
$ clojure -Fclojure.core/pr '{:a 1 :b 2}'
Key is missing value: {:a 1 :b 2}
or support both? #2020-07-31 21:14:12 borkdude not a big deal, but if the map is coming from a file, this could get a bit weird #2020-07-31 21:14:51 alexmiller because quoting sucks and it's more of a match to the -X #2020-07-31 21:15:24 borkdude quoting sucks, but I bet you will soon be quoting the individual args as well, there is no escape 😉 #2020-07-31 21:15:24 alexmiller but I implemented both at different points :) #2020-07-31 21:15:31 alexmiller no pun intended :) #2020-07-31 21:45:41 vlaaad powershell's -X:alias still broken 😞 #2020-07-31 21:46:19 vlaaad have you missed my suggested fix that handles splits to -X: and alias ?
if ($arg -eq "-X:") {
$sym, $params = $params
$ExecAlias += "$arg$sym", $params
} else {
$ExecAlias += $arg, $params
} #2020-07-31 21:57:46 alexmiller yeah, I haven't had a chance to look at that yet #2020-07-31 21:57:51 alexmiller it's in the queue! #2020-07-31 22:38:35 seancorfield @alexmiller I'm a bit surprised this first version fails (in 1.10.1.619):
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "87f4e2e6e9e4c7c3c2d4ccd3c8d7aad6d2b5d2cdb6c9"}, :content ("[email protected]")} #2020-07-31 22:42:02 seancorfield It's not a big deal but it worked before and I wasn't sure if it was a deliberate change or just an artifact of the implementation. #2020-07-31 22:49:57 alexmiller Yeah, that should probably work #2020-08-03 04:08:08 cfleming With the new :paths alias data feature, I’m assuming that those aliases are just fished directly out of the :aliases map in the (combined system/user/project) deps.edn map? i.e. there’s no alias selection done based on which aliases are passed by the user, or anything like that. #2020-08-03 04:44:56 alexmiller Right #2020-08-03 04:50:31 cfleming One further question: Cursive also shows aliases that the user can select in a tool window, which allows the user to decide which source paths etc should be synced to their project. I assume it doesn’t make sense to show these aliases there, i.e. it only makes sense to show those aliases whose value is a map. #2020-08-03 12:07:42 alexmiller Seems right #2020-08-03 04:09:28 seancorfield @cfleming That's my understanding based on playing with it a bit. #2020-08-03 04:10:31 cfleming Thanks @seancorfield. It gets a little confusing in the case of :extra-paths - those are calculated using the provided aliases, but then their alias values are taken from the raw map (I think). #2020-08-03 04:46:19 alexmiller In the latter question, aliases should be resolved from the merged edn map, same as paths. But maybe I’m not getting some nuance of your question #2020-08-03 04:46:58 cfleming Yep, that was what I had assumed. #2020-08-03 04:47:14 cfleming So using this map:
{:paths ["src" :clj-paths :resource-paths]
:aliases {:clj-paths ["src2"]
:resource-paths ["resources"]
:moar {:extra-paths ["src3" :moar-paths]}
:moar-paths ["src4"]}} #2020-08-03 04:48:05 cfleming src, src2 and resources will always be selected. src3 and src4 will be selected if the user has chosen to use the moar alias. #2020-08-03 04:51:46 seancorfield @U064X3EF3 It feels like this is part of a bigger picture that we don't have all the pieces of -- since the alias references in ;paths etc is all static at the moment, so you can't actually override any of it yet? #2020-08-03 12:03:30 alexmiller @cfleming right #2020-08-03 12:03:55 alexmiller @seancorfield not sure what you mean about overriding #2020-08-03 12:04:21 alexmiller This is about giving names to sunsets of paths #2020-08-03 12:04:28 alexmiller Subsets #2020-08-03 12:05:52 alexmiller One important place these names show up is in the classpath data in the basis #2020-08-03 16:54:23 seancorfield By "override" I meant that project aliases override user-level, and you can further override those on the command-line -- so I was thinking that you could provide a new :clj-paths alias on the command-line to "override" a :clj-paths alias in the project deps.edn. #2020-08-03 16:54:54 seancorfield Like I say, I'm still not seeing much benefit in that feature yet, so I assume there's a "bigger picture" and it will make more sense "soon" 🙂 #2020-08-03 17:22:18 alexmiller you could do all that but I do not have any use case in mind where that would be a good idea #2020-08-03 17:22:54 alexmiller better to manipulate with a -C alias modifying :extra-paths or overrides #2020-08-03 17:46:27 seancorfield Aye, and that's what I would tend to do -- so, again, I'm not seeing much benefit in being able to specify parts of paths etc as aliases right now. #2020-08-03 17:54:34 alexmiller then you should not do it :) #2020-08-03 04:11:05 seancorfield I guess you can supply new values via the command-line (`-Sdeps`) but I don't see much value in it as it stands. #2020-08-03 04:12:21 seancorfield I assume it's part of a bigger picture that we don't have all the pieces of yet... #2020-08-03 07:40:34 borkdude What does -S stand for? Just a random name? #2020-08-03 11:58:17 Janne Sauvala Based on all the comments I guess it stands for “something” 😁 #2020-08-03 12:07:56 alexmiller Setting #2020-08-03 12:10:26 borkdude Riiight #2020-08-03 12:30:44 alexmiller but "something" or "stuff" is pretty good too :) #2020-08-03 12:54:37 borkdude 🙂 #2020-08-03 12:54:57 borkdude Btw, I have wondered how you prevent :) turning into an emoticon. Is that a setting? #2020-08-03 12:55:28 borkdude Nm, I found it :) #2020-08-03 08:08:48 vlaaad I just went through deps guide and reference searching for words starting with "s". My guess is it means "Some option" or "Specify" #2020-08-03 11:08:27 dominicm I think it was script #2020-08-03 12:08:05 alexmiller Setting #2020-08-06 19:15:03 dharrigan I have a question, related to this extra-deps, using a couple of :local/root declarations, i.e., one that loads in rebel readline (with little customisation) and another which launches a nrepl (with some middleware injected). Can I still benefit from what the guide says? Can I compile the extra-deps includes, so that my clj startup is reduced? #2020-08-06 19:16:01 alexmiller sure #2020-08-06 19:16:14 alexmiller it's all on the same classpath #2020-08-06 19:17:02 dharrigan Okay, but where would the classes directory live, i.e., if I have something like this... #2020-08-06 19:17:05 dharrigan :rebel {:extra-deps {local-rebel {:local/root "/home/david/.clojure/libs/local.rebel" :deps/manifest :deps}
local-nrepl {:local/root "/home/david/.clojure/libs/local.nrepl" :deps/manifest :deps}}
:main-opts ["-m" "local.rebel.main"]} #2020-08-06 19:17:23 alexmiller I'd put it in its own alias and then use multiple aliases #2020-08-06 19:17:50 alexmiller ie, it's actually the same as what's in the doc, you just might have extra tool aliases in play #2020-08-06 19:18:18 dharrigan okay, let me have a play around. My clj startup at the moment is taking about 8 seconds to launch... #2020-08-06 19:29:50 souenzzo Oh....
Mine is ~2min on good machines
and we really don't care about that #2020-08-06 19:36:23 dharrigan :shocked_face_with_exploding_head: #2020-08-06 19:39:45 souenzzo I open the repl less then once a day.
(it remains open) #2020-08-06 19:18:21 alexmiller so clj -A:dev:rebel or whatever #2020-08-06 19:18:38 alexmiller or actually you've got main args in there #2020-08-06 19:18:54 alexmiller so you might want to do clj -R:dev:rebel when doing the compilation #2020-08-06 19:19:03 alexmiller so you get just the deps but not the main stuff #2020-08-06 19:19:16 dharrigan okay, will have a play 🙂 Thank you Alex! 🙂 #2020-08-06 19:19:27 alexmiller 🍺 #2020-08-06 19:19:39 dharrigan I'd love one thanks 🙂 #2020-08-06 19:38:43 seancorfield I only restart my REPL about once a week so even a few minutes startup wouldn't really bother me. I'm curious about workflows where startup times of eight seconds is problematic @dharrigan? #2020-08-06 19:40:10 dharrigan I have a need, a need for speed 🙂 I guess I'm just learning about if by applying some of the recommendations in that web page I can shave off some time! No harm in trying to optimise if I can 🙂 #2020-08-06 19:45:25 dharrigan Interesting, got it down to 3 seconds 🙂 #2020-08-06 19:59:10 alexmiller I'll send you my bill #2020-08-06 19:59:45 dharrigan Errrr....would an I.O.U suffice? #2020-08-06 20:00:22 dharrigan I.O.U one beer, fresh from the tap, if when you get to Ireland/UK 🙂 #2020-08-06 20:01:34 alexmiller I hope I can leave my house again someday #2020-08-06 20:23:47 lread @dharrigan’s avatar is a duck… he already has a bill! #2020-08-06 20:26:31 dharrigan He's the Egyptian Household God of Frustration. #2020-08-06 20:33:07 dharrigan Best Show Evar! #2020-08-06 20:25:17 seancorfield @alexmiller So I went and tried that dev startup time stuff (for the first time). I compiled a bunch of big libraries that my code relies on that rarely change, but did not compile my own code, then a ran a "load file" on my app's main file and it threw an exception about a clojure.core.cache.CacheProtocol not found... so I'm guessing there's a possibility for this pre-compile step to mess up in some way? #2020-08-06 20:26:03 alexmiller depends how you compiled everything #2020-08-06 20:26:46 alexmiller if you're trying to compile independent libs then you might not be compiling in the same order you actually load them in via your app #2020-08-06 20:27:52 alexmiller and then you can get into the case of recompiling a protocol, which can definitely go wrong #2020-08-06 20:27:57 seancorfield When I tried to repro with a (require 'api.main) instead of the load-file that the editor does, that seemed to work, so maybe it's something specific to load-file? #2020-08-06 20:28:39 alexmiller require will load too under the hood, but maybe #2020-08-06 20:29:49 seancorfield Ah, no difference. I started a fresh REPL, with those libraries precompiled and require failed this time, just as load-file had before. #2020-08-06 20:30:31 seancorfield So, you have to transitively compile "everything" to make this work, so that compile order doesn't matter? #2020-08-06 20:30:55 alexmiller transitively compiling everything should work (b/c it's exactly what your app does when it loads) #2020-08-06 20:30:59 seancorfield What about a project where you have multiple entry point nses, that represent separate applications? #2020-08-06 20:31:01 alexmiller other things might work :) #2020-08-06 20:31:28 alexmiller you could use more than one alias/classes dir if you find that to be a problem #2020-08-06 20:32:11 alexmiller or you could still use just compile with one and use from the other entry point. you might miss some stuff but presumably it's mostly the same in deps. #2020-08-06 20:32:28 seancorfield Currently, I start a REPL once a week (roughly) and just load code into it as needed so I usually end up with our entire code base loaded into the REPL eventually. #2020-08-06 20:33:08 alexmiller you are a bit outside the target user of this page :) #2020-08-06 20:33:31 seancorfield Compiling two different entry points that overlap might fall into the "recompiling a protocol" area? #2020-08-06 20:33:43 alexmiller depends #2020-08-06 20:34:37 alexmiller I am shorthanding the actual problem (recompilation itself is not the actual problem) #2020-08-06 20:35:39 seancorfield So the problematic piece (that surprises me a bit) is that (compile 'clojure.core.memoize) does not seem to transitively compile clojure.core.cache which I would have expected... I think that's the root of my earlier problem perhaps? #2020-08-06 20:36:41 alexmiller it should transitively compile anything loaded by clojure.core.memoize #2020-08-06 20:36:52 alexmiller but not everything in core.cache is used by memoize #2020-08-06 20:39:19 seancorfield When I look in the classes tree after (compile 'clojure.core.memoize) there are no core.cache classes at all. #2020-08-06 20:41:06 hiredman what about if you compile clojure.core.cache? my kind of guess would be it is already compiled/loaded somewhere else in your repl #2020-08-06 20:41:07 alexmiller any chance you've already loaded something that has loaded core.cache prior? #2020-08-06 20:41:27 alexmiller compile piggybacks load so if it's not being loaded (b/c it already is), it won't be compiled #2020-08-06 20:43:43 seancorfield Ah, that is a possibility, because something in my dev tooling might well be using that... If I explicitly (compile 'clojure.core.cache) I do see the .class files produced for it which would suggest it isn't loaded previously? #2020-08-06 20:43:56 seancorfield (or am I misunderstanding your point?) #2020-08-06 20:44:37 hiredman that is interesting, I wouldn't expect to see those if clojure.core.cache was previously loaded #2020-08-06 20:45:32 alexmiller I suspect you're getting the equivalent of :reload at the top level namespace you compile #2020-08-06 20:45:38 alexmiller but not on the ones below #2020-08-06 20:45:55 seancorfield Yup, confirmed. A "bare" REPL with just my dev tooling loaded has clojure.core.cache clojure.core.protocols clojure.core.server clojure.core.specs.alpha clojure.data.priority-map ... (non-`clojure.*` stuff omitted). #2020-08-06 20:46:39 hiredman ah, right, compile calls load-one directly, bypassing the already loaded check #2020-08-06 20:47:02 seancorfield So, yeah, I'd need to explicitly (compile 'clojure.core.cache) to make all this work, right? #2020-08-06 20:48:17 alexmiller you can do what's in the bottom of that guide (kind of), which is to address a similar problem (when user.clj has already loaded stuff) #2020-08-06 20:48:42 alexmiller (binding [*compile-files* true] (require 'clojure.core.memoize :reload-all))
#2020-08-06 20:49:05 alexmiller the only thing that makes load do compile is that dynvar #2020-08-06 20:49:13 alexmiller and the :reload-all will force a reload #2020-08-06 20:56:11 seancorfield I'm going to stick a list of (compile 'x.y.z) forms in a compile.clj file so I can do load-file on it to compile "everything" that I want pre-compiled and I'll add to it as/when I run into problems. FWIW, just with the set of libs I've put in that list so far, a cold (require 'api.main) has dropped from 18 seconds to 10 seconds. Like I say, since I only restart my REPL occasionally, that's really not a concern but now I'm curious about this idea of pre-compiling dependencies that rarely change... #2020-08-06 20:56:53 seancorfield ...and if a dependency does change, presumably you'd have to force a recompile of the newer version (and everything that depended on it) in order to pick those changes up? #2020-08-06 20:57:02 alexmiller you don't have to #2020-08-06 20:57:17 alexmiller a newer dependency should have newer .clj files and be preferred #2020-08-06 20:57:25 alexmiller it will just be slower till you recompile #2020-08-06 20:57:58 alexmiller clojure takes the newer .class or .clj #2020-08-06 20:58:32 alexmiller I guess if you changed deps to a newer version of a lib that was still older than your local .class files that wouldn't have that effect #2020-08-06 20:59:29 alexmiller I spent a half day trying to build this into clj so it could do it automatically (since it knows when your deps go stale). it was tricky for a bunch of reasons but maybe a future feature #2020-08-06 21:00:10 alexmiller the point where your .cpcache is stale is also the point when you want to recompile #2020-08-06 21:00:20 alexmiller so was trying to hook that #2020-08-06 21:00:35 seancorfield Yeah, discussions about it being a possible future feature of clj was what made me even curious to try it, once it was brought up here. #2020-08-06 21:03:14 alexmiller relevant for tools that replace the classpath too #2020-08-09 20:11:43 borkdude I get a ton of "DEPRECATED libs must be qualified" warnings not from my own deps.edn, but for transitive deps. Can I somehow turn that off? It's not something I can fix in my own codebase. #2020-08-09 20:12:29 borkdude Also one thought: would it make sense to only deprecate it for git deps and not for mvn deps, since mvn already has the foo = foo/foo implicit convention and there's not much tools.deps is going to change about that? #2020-08-09 20:23:07 vlaaad This is not about foo=foo/foo convention, this about a way of thinking about dependencies. Shorthands incentivize names squatting, full group/artifact names make you think who you depend on. I'd rather depend on borkdude/clj-kondo than on just clj-kondo. I'd rather released vlaaad/cljfx than just cljfx... I think it makes sense to state group/artifact for every procurer. #2020-08-09 20:23:34 dominicm I don't think maven does have that implicit thing, does it? #2020-08-09 20:23:44 dominicm They're very big on groups #2020-08-09 20:34:48 borkdude If you push a library foobar with lein to clojars, it's really foobar/foobar, so the group is foobar and the library is also called foobar. That's what I mean with a mvn library. The mechanism for pushing/retrieving is based on mvn. I know avoiding conflicts is important, but this convention has been there since ages in leiningen. Is this a lein convention or a mvn convention, I'm not sure. I haven't used pom.xml directly that much. #2020-08-09 20:36:42 borkdude So is this a design mistake of lein that deps now tries to solve? I totally understand the need for git deps to have a unique and predictable identifier. #2020-08-09 20:36:49 dominicm I suspect it's a lein and clojars thing. Maven central won't let you create an artifact like that, only clojars does (to controversy) #2020-08-09 20:40:01 borkdude Anyway, I hope that I turn off the warning for transitive deps, since I'm getting a ton of them:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "f0929f829b94859495b0bdb2a0c2c0c1c9"}, :content ("[email protected]")} #2020-08-09 20:44:03 seancorfield @borkdude You're only getting those for git deps -- not for all transitive deps. #2020-08-09 20:44:54 seancorfield If you were depending on released versions, you wouldn't get those warnings. I've only had to update my current project deps to get rid of warnings -- but I don't generally depend on git deps. #2020-08-09 20:46:57 borkdude I'm getting warnings for 1) my own git deps, 2) my own mvn deps, 3) mvn deps of transitive git deps. I'm not sure if a git dep depending on another git dep would also yield a warning, but I suspect this situation isn't that prevalent yet (let's call this 4).
I understand the need for 1. 2 is disputable, but from a "let's fix what lein did wrong" perspective, I can live with it. 3 and 4 I wouldn't find acceptable. #2020-08-09 20:47:12 borkdude 1 and 2 aren't shown in the output, I already fixed those. #2020-08-09 20:49:11 alexmiller You are getting these any time clj reads any deps manifest #2020-08-09 20:50:15 alexmiller So would for 4 as well #2020-08-09 20:52:06 alexmiller I could tailor these more, but this is a step towards removing support entirely #2020-08-09 20:56:03 alexmiller The thing I’m really trying to get rid of here is the need for canonicalization of libs when reading deps files #2020-08-09 20:56:47 seancorfield I have run across a handful of "single" named Java libs (`wsdl4j` plus a few others), but it's pretty rare in non-Clojure world. I agree that we've been very sloppy about that practice in the past (and I'm guilty a few times -- but I'm working to fix that 🙂 ) #2020-08-09 20:56:59 borkdude Well, leaving it as it is now will probably force people to update their deps.edns quicker ;) #2020-08-09 20:57:33 alexmiller Well that’s the idea :) #2020-08-09 21:01:55 dominicm I'm glad there's more of a push on this. I've seen a few conflicts in libraries (I've seen at least two libraries using roll.core).
I've gotten friction in the past because it's "verbose", so it's nice this will become non optional. #2020-08-09 21:02:44 borkdude I think that example just shows how .core doesn't contribute to avoiding conflicts while still having a multi-segmented name #2020-08-09 21:04:03 seancorfield I think .core is a horrible default 😐 #2020-08-09 21:06:29 seancorfield That's why clj-new doesn't do that. You must provide either a group-id/artifact-id project name (and the README talks about using reverse domain names or other identity-based groups) or you can provide a multi-segment namespace name. But I may downplay that option a bit more, now that t.d.a. is "encouraging" proper group/artifact lib names 🙂 #2020-08-09 23:28:52 cfleming @alexmiller -Stree seems to be broken since 1.10.1.624. It’s because previously root deps used to return nil in their :dependents set, and now they return an empty collection. This broke Cursive, I’ve fixed it there by inlining make-tree and changing:
(let [{roots false, nonroots true} (group-by #(-> % val :dependents boolean) lib-map)]
to:
(let [{roots false, nonroots true} (group-by #(-> % val :dependents empty? not boolean) lib-map)] #2020-08-09 23:30:39 cfleming I’m not sure what the cause of that change is, I couldn’t see anything immediately suspicious in the changelog but I only had a fairly cursory look. #2020-08-09 23:32:06 cfleming On a related note, is the :dependents/`:children` info returned from resolve-deps considered to be part of the API and thus something I can rely on? It’s not mentioned in the docstring. #2020-08-09 23:32:06 cfleming On a related note, is the :dependents/`:children` info returned from resolve-deps considered to be part of the API and thus something I can rely on? It’s not mentioned in the docstring. #2020-08-09 23:43:31 seancorfield The output of -Stree has certainly changed -- I just updated from .619 to .641 and my outdated deps script broke (so that'll be a bit of fun for me to debug and fix tomorrow morning when I get to work!). #2020-08-09 23:43:31 seancorfield The output of -Stree has certainly changed -- I just updated from .619 to .641 and my outdated deps script broke (so that'll be a bit of fun for me to debug and fix tomorrow morning when I get to work!). #2020-08-09 23:43:50 cfleming Well, it’s changed from appearing to not appearing at all. #2020-08-09 23:44:32 seancorfield Ah, yes, that would definitely break my script. I hadn't actually tested basic output yet. #2020-08-09 23:44:38 cfleming ~/d/c/issues (master)> clj -Stree
~/d/c/issues (master)>
#2020-08-09 23:45:18 seancorfield Oops. #2020-08-09 23:45:36 seancorfield Well... I guess I don't need to debug my script 🙂 #2020-08-10 01:47:33 alexmiller I actually rewrote most of the tree stuff this morning, still working on it though #2020-08-10 02:24:20 alexmiller @cfleming ah, I see where I broke it, will fix. I don't know who's updating the homebrew core tap, but it's just random versions now #2020-08-10 02:26:25 cfleming @alexmiller Ok, I basically copied your make-tree function and fixed my copy in Cursive. As long as I can rely on the :dependents/`:children` bit coming back from resolve-deps that’s probably a better idea anyway, since I won’t have to fix anything if you change make-tree to do something different. #2020-08-10 02:30:52 alexmiller the bug was in the last refactoring of lib-paths #2020-08-10 02:31:47 alexmiller so basically in the guts of resolve-deps #2020-08-10 02:34:18 cfleming Is dependents enough to build the tree without children? #2020-08-10 02:34:50 alexmiller depends what you're trying to do with it #2020-08-10 02:35:04 alexmiller make-tree is going to be moving and changing quite a bit - are you using that directly? #2020-08-10 02:35:14 cfleming Basically, show the user a dependency tree in Cursive. #2020-08-10 02:35:43 cfleming I was, but after working around this bug this morning I’m now using a fixed copy of the old version. #2020-08-10 02:36:20 alexmiller this whole thing with you depending on whatever arbitrary version the user happens to have is crazy talk man #2020-08-10 02:37:00 cfleming That’s why I want data I can depend on in the resolve-deps results (or anywhere else, don’t care where I get it from) that I can use to build a tree to show the user. #2020-08-10 02:37:50 alexmiller the reason I'm rewriting make-tree is I suspect the same reason you're relying on that stuff - to better form the tree and the lib-map not being sufficient #2020-08-10 02:38:03 cfleming Basically, until there is a defined API that gives me what I need I have no choice but to hack the guts. #2020-08-10 02:38:15 cfleming Which is intrinsically tied to concrete versions. #2020-08-10 02:38:50 alexmiller well, depending on specific concrete versions is fine :) it's having no control over what you're using that seems like crazy talk #2020-08-10 02:39:07 cfleming I can never control what the user has installed. #2020-08-10 02:39:29 cfleming I can let them choose the version that they want to use, but I can’t mandate versions. #2020-08-10 02:39:30 seancorfield @cfleming What are you actually using this information for? #2020-08-10 02:39:47 alexmiller I guess I don't see why you can't use a specific version of tools.deps independent from whatever clj they have installed #2020-08-10 02:40:22 cfleming Because when you introduce new stuff in deps, people aren’t going to like having to update Cursive to get it. #2020-08-10 02:40:43 alexmiller seems better than having it break every time I update clj #2020-08-10 02:40:46 cfleming And as long as there’s a stable API, it shouldn’t matter. #2020-08-10 02:40:52 alexmiller there's not a stable API #2020-08-10 02:40:56 alexmiller that's the alpha part #2020-08-10 02:41:04 cfleming Then it will break. #2020-08-10 02:45:11 cfleming If the user finds a particular version of deps breaks with a particular version of Cursive, they can choose the version of deps they want to use within Cursive, or they can downgrade their CLI until either Cursive or deps gets fixed. #2020-08-10 02:46:09 seancorfield I'm continually impressed at the work you do with Cursive but at the same time it just reinforces why I like a simple editor and simple command line tools 🙂 #2020-08-10 02:46:09 seancorfield I'm continually impressed at the work you do with Cursive but at the same time it just reinforces why I like a simple editor and simple command line tools 🙂 #2020-08-10 02:46:29 cfleming ed will always be there for you. #2020-08-10 02:46:33 cfleming 🙂 #2020-08-10 02:48:51 seancorfield You can pry Atom/Chlorine and the Clojure CLI from my cold, dead hands mate! #2020-08-10 02:49:22 seancorfield (although I've recently taken to using Reveal instead of REBL, just as an experiment with my workflow... but I'm missing several of the REBL features) #2020-08-10 02:50:07 cfleming To be fair, I think you have a broken script until the CLI is fixed. #2020-08-10 02:50:40 alexmiller fixed in clj 1.10.1.645 #2020-08-10 02:50:52 alexmiller which is uploading right now #2020-08-10 02:52:39 seancorfield looks fixed -- thanks @alexmiller #2020-08-10 02:55:58 seancorfield And my outdated deps script works again -- yay! #2020-08-10 03:00:53 alexmiller @cfleming I'll give you a heads up before I publish the tree stuff - still working on it. #2020-08-10 03:01:03 cfleming Ok, thanks. #2020-08-10 03:01:50 alexmiller just as background, deps expansion does not actually ever build an explicit tree during expansion and in fact it's not a tree, it's really a graph. #2020-08-10 03:02:53 alexmiller the make-tree/print-tree stuff is kind of retroactively trying to build a tree from what's in the lib map, but that's really also not an explicit tree and does not contain sufficient path info to respect the exclusions the user originally stated #2020-08-10 03:03:40 alexmiller I've rebuilt the tree stuff to work from the trace (which is actually returned in the metadata on the lib map if it was requested), but it's surprisingly complicated to rebuild a tree that matches the user's original intent #2020-08-10 03:04:16 cfleming I can believe that. #2020-08-10 03:04:40 cfleming Mostly I’d just like a stable way to replicate what you show the user on the command line in Cursive. #2020-08-10 03:05:22 alexmiller yeah, and there will be a function somewhere that can work from the trace (which you can actually emit from clj with -Strace) to give you that tree data #2020-08-10 03:05:23 cfleming I mean, I can actually show them a graph if required. #2020-08-10 03:05:57 alexmiller that's kind of the stuff I'm wrestling with - there are multiple things you might want to show #2020-08-10 03:06:38 alexmiller what most of the deps tools show is basically a tree with every lib listed once (even though it's really possibly a dep at many points) #2020-08-10 03:07:22 alexmiller but you in particular might really want the "full" tree - full expansions at every node (could still stop at cycles in a path) #2020-08-10 03:08:01 alexmiller and then for debugging you really want to know - what version was requested, what version was actually selected, and what libs were excluded at which different points #2020-08-10 03:08:46 cfleming Yup, and then I could collapse duplicates to mirror what users are used to seeing in other tools, while still allowing them to expand them if required. #2020-08-10 03:09:09 alexmiller all of that info is in the version map included in the trace but it requires a fair amount of trawling #2020-08-10 03:10:57 cfleming When you talk about the trace, do you mean something included in the new basis stuff? #2020-08-10 03:11:05 alexmiller no, pre-dates that actually #2020-08-10 03:11:18 alexmiller clj -Strace will dump a trace.edn file #2020-08-10 03:12:07 cfleming Is that documented anywhere? It doesn’t seem to be in the reference doc. #2020-08-10 03:12:44 alexmiller no, I wasn't sure how stable it was going to be. I added that when I made tools.deps.graph so basically prior to my conj talk #2020-08-10 03:13:22 alexmiller but it hasn't actually changed since then I don't think and this tree work will add another "user" of it #2020-08-10 03:14:07 cfleming Ok, looks like I could use that if the generation of it were available in t.d.a and I could call that. #2020-08-10 03:15:09 alexmiller it has been available in t.d.a since since last November in the return of resolve-deps #2020-08-10 03:15:26 alexmiller if you pass the :trace option, it will be included in the metadata of the lib map that's returned #2020-08-10 03:16:06 cfleming Ok. If that’ll be a more stable thing for me to use I’ll start using that. #2020-08-10 03:16:32 alexmiller no plans to change it ;) #2020-08-10 03:17:14 cfleming What’s missing from the doc is descriptions of what’s actually in the trace log, or the runtime basis for that matter. Are they specced or something? #2020-08-10 03:17:34 alexmiller no, I wasn't quite ready to commit to them yet and haven't really caught up on that #2020-08-10 03:17:53 alexmiller the basis is doc'ed in calc-basis #2020-08-10 03:18:55 alexmiller it is all a bit strewn around atm and specs are missing some of those #2020-08-10 03:19:28 alexmiller so, definitely some doc debt to pay back. my time is kind of split right now with joining nubank :) #2020-08-10 03:20:25 cfleming Ok, when I get some time I’ll investigate switching to that, and falling back to what I do now if the metadata isn’t available. I’ll be interested to see your tree code when it’s done. #2020-08-10 11:22:38 pez Anyone know of a docker image containing both leiningen and clojure? I’m migrating a project piece by piece and am using the lein-tools-deps plugin. Currently using jdk 8, as well, unsure how hard that dependency is though. #2020-08-10 11:22:38 pez Anyone know of a docker image containing both leiningen and clojure? I’m migrating a project piece by piece and am using the lein-tools-deps plugin. Currently using jdk 8, as well, unsure how hard that dependency is though. #2020-08-14 18:33:39 cap10morgan The clojure:latest image now has both lein and CLI tools installed if that helps. #2020-08-10 11:37:43 borkdude @pez FWIW I usually just install clojure in a Docker container myself #2020-08-10 11:37:59 borkdude See any of my latest projects #2020-08-14 18:32:34 cap10morgan Is this a bug? When I update my pom.xml file with clojure -Spom using http://1.10.1.itit is nesting another <build> element under the existing one. So my diff looks like this:
<build>
- <sourceDirectory>src</sourceDirectory>
+ <build>
+ <sourceDirectory>src</sourceDirectory>
+ </build>
</build>
#2020-08-14 18:33:39 alexmiller it is a bug #2020-08-14 18:33:55 alexmiller has been fixed in a subsequent prerelease but not yet in a stable one #2020-08-14 18:34:05 cap10morgan ah ok. cool, thanks! #2020-08-14 18:34:58 alexmiller it's fixed as of 1.10.1.590 (latest prerelease is 1.10.1.645) #2020-08-14 18:35:22 alexmiller or you could fall back to older stable #2020-08-14 18:35:47 seancorfield Which prerelease introduced the bug where it overwrote the groupId? (since fixed) #2020-08-14 18:36:58 cap10morgan exactly what I was wondering 🙂 #2020-08-14 18:37:02 alexmiller I think it was 1.10.1.590 #2020-08-14 18:37:43 alexmiller which was fixed in 1.10.1.636 #2020-08-14 18:37:57 alexmiller it is hard to keep track of all these across the two repos #2020-08-14 18:38:29 alexmiller but I would say if you're going to use any prerelease, I would use the latest #2020-08-14 18:40:07 alexmiller I have 3 or 4 pending things to get into clj and then I'm hoping to get to another stable release #2020-08-16 12:36:09 borkdude E.g.:
clojure -A:depstar -m hf.depstar.uberjar MyProject.jar -v -m proj --classpath "$(clojure -Spath)"
#2020-08-16 12:36:57 borkdude Btw, I think depstar will be able to compile with GraalVM so maybe I can even include it in babashka itself to create babashka uberjars #2020-08-16 16:46:46 Alexis Vincent I’d like to distribute a deps.edn file of aliases for everyone to use in our company. But I want people to be able to have their own local ./clojure/deps.edn. Is there a way to specify a deps.edn file to merge in, either globally for all projects, or from a projects deps.edn? #2020-08-16 16:58:18 alexmiller no, not currently #2020-08-16 17:29:28 j0ni Is it possible to add a list of jar files from a project sub-dir to the classpath without having to name them? I know about :local/root and I see this in the docs https://clojure.org/guides/deps_and_cli#local_jar but I have a lot of them and they aren't published anywhere. Do I just have to make up names for them and add them individually as :deps? #2020-08-16 17:32:33 alexmiller It’s kind of a hack, but java supports on the classpath for this (literal path/ so not shell expanded). I think if you added a :paths entry for “path/*” that should work #2020-08-17 08:15:49 rickmoynihan Interesting I was wanting to do something similar too, so I could deploy an app as a zip with a directory of jars in lib but start it with the clojure command line tooling; via a :local/root; with a different deps.edn for production, rather than the one we use in dev. #2020-08-17 13:43:08 alexmiller the other crazy thing you can do is to reference jars in your manifest (this was part of the original applet support) with the Class-Path attribute but I think that has to be explicit jars. afaik that does not support the glob stuff #2020-08-17 14:55:34 rickmoynihan yeah now you mention it I’ve seen manifests do this…. a long time since I’ve seen or written an applet tho! 😆 #2020-08-17 14:57:06 alexmiller also relevant, the manifest has a line wrapping limit and so other arcane rules so it's best to use the apis to make it rather than doing it by hand #2020-08-17 14:59:06 rickmoynihan wow I had no idea… eitherway if I do this I think it’ll be via the :local/root method, largely motivated via the new -F -X stuff #2020-08-16 17:32:58 j0ni thank you @alexmiller I will try that out #2020-08-16 17:59:13 j0ni amazing, thank you Alex, it works #2020-08-16 19:30:50 borkdude @seancorfield Awesome: depstar works with GraalVM and can hence be included in babashka proper so it can make its own uberjars #2020-08-16 19:34:36 borkdude $ bb -cp $(clojure -Spath) --uberjar foo.jar -m proj
Building uber jar: foo.jar
$ bb -jar foo.jar
:foo - failed: int?
#2020-08-16 20:24:13 seancorfield @borkdude Cool. As for your earlier question, depstar deliberately just includes everything on the current classpath, aside from itself. #2020-08-16 20:26:28 ghadi That was a fundamental design decision I regret #2020-08-16 20:27:13 ghadi In retrospect I’d use t.d.a as a library to calculate what to tar up #2020-08-16 20:27:37 ghadi rather than subtracting depstar itself from the classpath #2020-08-16 20:29:21 borkdude yep. but that can be easily changed. I've already done that in my fork #2020-08-16 20:31:28 ghadi You using t.d.a to calculate instead? #2020-08-16 20:32:26 seancorfield To be honest, using the classpath has made depstar a lot easier to use for people, and a lot more predictable. #2020-08-16 20:33:06 borkdude @ghadi I just added a :classpath option that can be used to override #2020-08-16 20:34:00 seancorfield Hmm, interesting approach. Since it's easy to get clojure to produce an arbitrary classpath from deps.edn aliases etc... #2020-08-16 20:34:10 borkdude Exactly. #2020-08-16 20:36:08 ghadi this is a similar thing to what I’m trying to avoid, vectors of strings #2020-08-16 20:36:54 borkdude So the classpath given to babashka, produced by clojure is just passed down to depstar:
$ bb -cp $(clojure -Spath) --uberjar foo.jar -m proj
which results in a foo.jar that can be invoked with bb -jar foo.jar. #2020-08-16 20:38:43 ghadi I understand. I’m just saying the shell is not leverage compared to a clojure library. One of the things I want to do is split a list of dependencies by their rate of change for smarter distribution to containers or Lambdas #2020-08-16 20:41:01 borkdude You mean, make uberjars for things not changing often? #2020-08-16 20:42:08 ghadi For containers, you don’t want every commit to result in a 60MB new uberjar, you want some structural sharing across commits #2020-08-16 20:42:15 borkdude I guess one could use the -A or -T option for that, to make groups of deps and then push that classpath into depstar #2020-08-16 20:42:57 ghadi Yeah I know you can hack it in shell, but why not program it directly in bb or clojure? #2020-08-16 20:43:34 ghadi my hope with tools.build (not that I have anything to do with it currently) is that it encourages build tasks to compose better than the lowest common denominator of the CLI #2020-08-16 20:43:54 borkdude bb will have an --uberjar option, but it can also have a bb -e '(babashka.uberjar/run {:classpath ...})-ish interface #2020-08-16 20:44:17 borkdude can have them both #2020-08-16 20:44:36 vlaaad Wait are tools.build out? #2020-08-16 20:44:49 ghadi no #2020-08-16 20:45:13 seancorfield depstar currently has zero dependencies, except Clojure itself, so it can't interfere with anyone's project dependencies -- that's worked very well in practice. I like @borkdude’s idea of just giving it an explicit classpath to build form 🙂 #2020-08-16 20:47:40 vlaaad Unless I want to have the whole build happen in a single vm, and it has to have other build-specific dependencies like data.xml for updating pom.xml. I use 2 vms for that now, but I'd rather not :) #2020-08-16 21:02:32 seancorfield I'm not sure whether you're agreeing with depstar's approach or suggesting something different? #2020-08-16 20:45:33 borkdude bb won't do AOT btw (maybe it could but I don't think it's worth it). for JVM uberjars (with AOT-ed classes), just use clojure / depstar #2020-08-16 20:45:53 borkdude bb uberjars typically will contain sources and other assets, but no classes #2020-08-16 20:47:46 borkdude I did have depstar compile with GraalVM as is and when I didn't remove the AOT-related code, it did compile, but the binary got somewhat bloated (+ 30MB, happens when it hits a specific part of Clojure). So maybe compile just works natively, since it's just going to transform .clj into classfiles, I haven't tried it. #2020-08-16 20:51:04 borkdude I'll try it now... just for science #2020-08-16 21:15:30 borkdude Interesting. Compilation of a class doesn't work in a GraalVM binary because the classpath inside the binary is not set correctly for clojure RT. I guess it won't work for that reason, although technically it could maybe work. #2020-08-16 21:24:27 borkdude Going to try this:
(binding [*compile-path* (str tmp-c-dir)]
(clojure.lang.RT/compile (str main-class ".clj"))) #2020-08-16 21:25:51 borkdude crap, it's private ;) #2020-08-16 21:38:31 borkdude With a reflective hack it works. I see all kinds of AOT-ed sourced, even aot__init.class, but no aot.class. Maybe :gen-class isn't part of RT/compile? Eh.... enough for today. #2020-08-16 21:42:01 borkdude So the AOT part could be useful when depstar had a binary CLI version, so to speak. I don't need it for bb, but it could be useful for fast build tooling in general. #2020-08-16 21:45:58 borkdude To make this work clojure.lang.RT should also have a pluggable classpath resolver. It might work for Clojure files, but I wonder how it will work if code relies on .class files from the Java world. Does the Clojure compiler kind of treat .class files as data, without actually having to have them on the classpath per se, as long as you point it to the right file? I.e. no runtime need, just able to read and analyse statically? #2020-08-16 22:05:12 borkdude I see, Compiler relies on reflection when dealing with interop, so there it will likely break, since the reflection API only works with classes that are part of the same runtime. OK, this was a nice tangent. #2020-08-17 08:56:34 borkdude Creating uberjars with depstar native in bb now also works on Windows: #2020-08-17 13:40:25 jimberlage Anyone ever seen a message like the following after updating the :sha of a git dependency:
Error building classpath. Destination path "my-lib" already exists and is not an empty directory
org.eclipse.jgit.api.errors.JGitInternalException: Destination path "my-lib" already exists and is not an empty directory
#2020-08-17 13:43:14 delaguardo you can delete local artifacts in $HOME/.gitlibs/{libs,_repos} and restart the repl #2020-08-17 13:44:09 delaguardo I don’t know exactly what cause that problem. In my case it happens after changing :git/url #2020-08-17 13:44:53 alexmiller I believe this is the result of parallel downloading to gitlibs messing up one of the cached git dirs #2020-08-17 13:50:52 alexmiller if you clear your ~/.gitlibs (either fully or selectively) and then do clj -Sforce -Sthreads 1 that should force a re-download singlethreaded #2020-08-17 13:51:16 alexmiller we have a ticket for this, I haven't had a chance to really look at in depth yet #2020-08-17 13:55:58 jimberlage Thanks! Trying that revealed another error - technically auth-related, but testing ssh pushes/pulls to github on this repo still works for me, so I'm wondering if this is somethimg on my end or a symptom of a different bug:
Error building classpath.
#2020-08-17 13:59:12 alexmiller you'll need to debug why it's not finding the key:
1. is ssh-agent running? if not, start it
2. does ssh-add -l list your public key? if not, add it
3. if you're on a Mac, check whether your ~/.ssh/config has Host * / IdentityFile in it, if so comment that out or remove it #2020-08-17 13:59:48 alexmiller after that, things get trickier to debug, could be key type #2020-08-17 14:05:43 jimberlage Looks like the bit about the IdentityFile line fixed it. Thank you #2020-08-17 14:14:03 alexmiller 👍 #2020-08-19 11:10:25 borkdude Why does clojure not warn about an unexisting :test alias?
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "f99b968b929d8c9d9cb9b4bba9cbc9c8c0"}, :content ("[email protected]")}
#2020-08-19 11:11:28 borkdude Also, I figured it would be useful to have something like: clojure -CR:test to get the same as clojure -C:test -R:test: use case, I want to run a REPL with the same paths and deps as the :test alias, just not execute the main #2020-08-19 13:11:29 alexmiller you can use clojure -A:test to do that #2020-08-19 13:12:07 alexmiller oh, "and not execute the main" - well for that you need the -C / -R #2020-08-19 11:28:25 vlaaad I think :test exists in install deps? #2020-08-19 11:58:11 dharrigan indeed, in my install on Arch, in /usr/share/clojure/deps.edn it has test as an alias #2020-08-19 17:30:05 alexmiller -Scp is an existing clj option btw #2020-08-19 17:40:19 borkdude That's useful, but it only affects the runtime classpath. This feature is about overriding the classpath from which the uberjar is created, which makes it more flexible. The classpath can be generated from other tools like leiningen, boot, clojure, etc, whatever source. #2020-08-19 17:46:36 alexmiller gotcha, might be something good to add to tools.build's uber stuff then :) #2020-08-19 19:41:26 Janne Sauvala @alexmiller I keep hearing references to tools.build (since your Inside Clojure -post). Is there more detailed description of it somewhere already or can we expect something coming out soonish? 🙂 #2020-08-19 20:01:12 seancorfield I suspect the answers are "no" and "maybe" -- depending on your definition of "soon" 🙂 #2020-08-19 20:09:15 alexmiller ^^ #2020-08-19 20:09:38 alexmiller trying to pull together a lot of strands right now, not sure on timing yet #2020-08-20 05:59:19 Janne Sauvala Gotcha. I’m exited to hear tools.build could be a thing. I really enjoy working with tools.deps and the latest features look great 👍:skin-tone-2: #2020-08-19 17:30:42 alexmiller we would definitely like to encourage more use of -X - considering a bunch of additional features that may continue to make it more attractive for tools #2020-08-19 17:52:14 vlaaad Features like...? :) #2020-08-19 17:59:03 alexmiller well, still designing :) #2020-08-19 18:01:26 alexmiller but really trying to simplify invocation in several ways #2020-08-19 18:39:16 seancorfield Updating hf.depstar.uberjar:
(defn run
"Generic entry point for all invocations.
Can be used with `clojure -X`:
In `:aliases`:
:depstar {:extra-deps {seancorfield/depstar {:mvn/version ...}}}
:uberjar {:fn hf.depstar.uberjar/run
:args {:aot true :jar-type :uber}}
Then run:
clojure -R:depstar -X:uberjar :jar MyProject.jar :main-class project.core
If the destination JAR file and main class are fixed, then could be
added to `:args` in `deps.edn`:
:depstar {:extra-deps {seancorfield/depstar {:mvn/version ...}}}
:uberjar {:fn hf.depstar.uberjar/run
:args {:aot true :jar MyProject.jar
:jar-type :uber :main-class project.core}}
Both `:jar` and `:main-class` can be specified as symbols or strings."
It would be nice if -X acted like -R .. -X .. so you didn't have to also specify -R:depstar here and could have the :extra-deps in the :uberjar alias. #2020-08-19 18:41:39 alexmiller what I'm working on right now is in that area #2020-08-19 18:42:21 alexmiller all over it! #2020-08-19 17:30:48 alexmiller and easier to use #2020-08-19 17:42:34 borkdude @seancorfield One other thing that came up in #beginners: the non-standard way of providing the name of the uberjar. It would be easier to compose if that could be done using a named parameter like --uberjar foo.jar #2020-08-19 18:40:59 seancorfield @borkdude Yeah, I'm making it :jar for the -X option and it will also be -J / --jar on the command-line (as well as just accepting the filename anywhere). #2020-08-19 18:47:14 dakra Nice :) #2020-08-19 18:41:25 seancorfield So @daniel415’s example will work exactly as he would like. #2020-08-19 18:41:37 borkdude 👍 #2020-08-19 20:17:22 seancorfield OK, in light of earlier discussions, the develop branch of depstar has several new features requested this morning: support for clojure -X, support for overriding the classpath used for building the JAR, and reworked argument parsing so the JAR filename can be specified "anywhere" (before/after/in the middle of the options). Please take it for a spin and report back any issues (either on GitHub or here). seancorfield/depstar {:git/url "" :sha "c461ba830e069edf0f011169c18595a4a8f917fc"} Thanks in advance #2020-08-20 12:41:14 timo 👋
How do I pass a version to clj -Spom? #2020-08-20 12:43:01 timo I need it for CI. or is there a more clever way to update the version in the pom.xml for clojars-deployment? #2020-08-20 13:15:42 timo sure, this is another option #2020-08-20 13:15:44 timo thanks #2020-08-20 12:48:27 dominicm @timok generally you cannot. clj -Spom only updates your deps (and generates a basic outline). #2020-08-20 12:48:49 dominicm There's garamond which can do that though #2020-08-20 12:50:09 jose @timo you can use the maven versions plugin: mvn versions:set versions:set-scm-tag -DnewVersion="${version}" #2020-08-20 12:51:08 timo ah thanks @dominicm @jlle. then I will keep the pom.xml in the repo. #2020-08-20 13:09:14 timo thanks @jlle but it's actually circleci but I will have a look 👍 #2020-08-20 14:41:19 jose Do you think that the new -T CLI option can be a better alternative to user.clj? I can imagine moving some repl helpers I have in my user.clj to a function and invoke that function explicitly with -T #2020-08-20 14:47:39 alexmiller no, that doesn't seem right #2020-08-20 14:48:02 alexmiller if you're starting a repl, you can use the clojure.main -i to do things like that #2020-08-20 14:48:53 alexmiller -T is for invoking external tools (although I think -T is actually going to go away anyways) #2020-08-20 16:06:45 seancorfield Was -T intended to be a specialized -R/`-C` combination? #2020-08-20 16:07:39 seancorfield I was looking on the CLI reference page for where you can use :deps in an alias and that seems to just be -T -- but I thought it was valid in some other use case...? #2020-08-20 16:07:46 borkdude I think -T did not include the deps on the top level. Which I can see being useful. Kind of like a system wide alias, but for projects #2020-08-20 16:10:04 seancorfield It feels "simpler" to just allow :deps / :paths in any alias with the expectation that it overrides the top-level values... but I guess there's always the question of what happens if you specify two aliases and the first has :deps / :paths to override and then the second alias has :extra-deps / :extra-paths to add to those? Will people expect that to work on will :deps / :paths always "win"? #2020-08-20 16:10:05 alexmiller :deps/paths replacement was previously only available via -A, was pulled out as -T in past #2020-08-20 16:10:16 alexmiller we are reworking all of this stuff right now #2020-08-20 16:10:33 seancorfield Too many unanswered questions 🙂 Needs more Hammock Time! #2020-08-20 16:10:51 alexmiller already did it #2020-08-20 16:10:55 borkdude paths replacement = classpath overrides? #2020-08-20 16:11:01 alexmiller no #2020-08-20 16:11:11 alexmiller :paths replaces the :paths in your project deps.edn #2020-08-20 16:11:16 borkdude right #2020-08-20 16:11:22 alexmiller which is admittedly a very unusual use case #2020-08-20 16:11:34 borkdude can be useful for a project within a project? #2020-08-20 16:11:39 alexmiller we're going to rename these to :replace-deps and :replace-paths #2020-08-20 16:12:03 borkdude yeah, that's clearer #2020-08-20 16:12:23 seancorfield And if there are multiple of those, it'll be like :main-opts and only one gets picked? #2020-08-20 16:12:52 alexmiller it's not worth me going through all this by piece - it's all getting updated #2020-08-20 16:13:02 alexmiller so let me go work on that :) #2020-08-20 16:13:09 borkdude an analogy with :classpath-overrides {foo/bar nil} could be :remove-deps and :remove-paths #2020-08-20 16:15:45 seancorfield FWIW, I'm liking the -X stuff (modulo always wanting :extra-deps to be respected by it) and I'm thinking it will simplify several pieces of tooling at work, since we can remove multiple namespaces containing just a -main function that exists solely to parse/route arguments to other functions that take a hash map 🙂 #2020-08-20 16:16:18 alexmiller that's the idea (and usage stuff will be improved) #2020-08-20 16:16:45 seancorfield (our current problem is a couple of our servers are too old to be able to run the install script so I have to update them manually -- but that's not Clojure's problem 🙂 ) #2020-08-20 16:16:58 alexmiller also working on ideas to make ad hoc -F variant a bit more useful for the case of one namespace with many entry point fns #2020-08-20 16:17:12 alexmiller have I talked about -F yet? I can't even remember now #2020-08-20 16:17:28 borkdude no, go on ;) #2020-08-20 16:17:46 seancorfield -F is like -X only without needing an alias 🙂 #2020-08-20 16:18:00 alexmiller yeah, invoke ad hoc function #2020-08-20 16:18:01 borkdude why even have X then? #2020-08-20 16:18:12 alexmiller lets you package default args and not repeat the fn at cli #2020-08-20 16:18:37 alexmiller both useful in different contexts #2020-08-20 16:18:42 seancorfield Invoking via an alias is more flexible. Yeah, default args, simpler name, will be even more useful when it respects :extra-deps 🙂 #2020-08-20 16:18:56 borkdude you can stuff default args in the function itself though? #2020-08-20 16:19:29 seancorfield Sure, but you could have different aliases with different default args. That's a use case that we could leverage at work. #2020-08-20 16:19:37 borkdude -A:foo -F foo/bar, respects extra deps of foo alias? #2020-08-20 16:20:35 seancorfield Currently, you'd need -R:foo -X:foo for a function alias -- repetitive! -- so if -X respects :extra-deps you could just do -X:foo #2020-08-20 16:22:20 alexmiller this is coming #2020-08-20 16:23:31 seancorfield Yeah, at this point I'm just sort of assuming it will do that 🙂 #2020-08-20 16:21:46 seancorfield But -F is definitely useful for ad hoc command-line work. #2020-08-20 16:22:02 borkdude instead of creating different aliases with different defaults, one could also just write multiple functions in one namespace. I don't directly see the benefit of one over the other, -A:foo -F foo/foo, -A:bar -F foo/bar. #2020-08-20 16:22:50 alexmiller sure if you bake it into the code #2020-08-20 16:23:01 alexmiller putting it in deps.edn is default but configurable #2020-08-20 16:23:12 borkdude I guess having it closer together is one benefit #2020-08-20 17:04:58 dominicm -F seems like -e? But with less ( #2020-08-20 17:07:37 alexmiller -e is arbitrary expression, -F is "invoke a functional entry point that takes a map" so it's much more specific #2020-08-21 01:59:26 steveb8n Q: Is this a silly idea? I want a combination of git deps and “-LATEST” behviour for my CI/test builds. For prod builds I want static git shas but for automated/test builds, it would be great to automatically use the most recent git sha. Is this even possible? #2020-08-21 02:35:07 souenzzo Temporal-basead versions are always bad IMHO
If your CI fail and you run it again, it may take a new commit and success again. So success and fail says nothing.
Pass the commit ID to your CI or fetch and use a sed / template.
Then in case of success/fail, you will know which code succeed/failed #2020-08-21 02:35:28 souenzzo And you will be able to run again the same CI in a older commit/retry #2020-08-21 02:42:56 steveb8n thx. fetch and sed is an interesting idea. I could even use bb for this script. #2020-08-21 02:43:36 steveb8n how would the script apply the latest sha? by writing it into the deps.edn? or is there some other trick? #2020-08-21 02:45:41 steveb8n fyi: the reason I want this is to avoid the endless commits I currently do for all my git deps during dev/test. but I really like using git deps in CI #2020-08-21 02:55:23 seancorfield Running CI and production against different versions seems like a recipe for hard-to-debug disaster... #2020-08-21 02:56:35 seancorfield I can understand wanting your local dev-only tooling to always pick up the RELEASE (or perhaps occasionally the LATEST) release but I really don't want any variance in versions between dev, CI, production for anything I depend on. #2020-08-21 02:57:01 seancorfield (so I guess my answer to your actual Q would be "yes, it's a silly idea" IMO) #2020-08-21 03:19:50 steveb8n that is useful perspective. I’ll apply a hammock to this a little more. thanks #2020-08-21 03:24:27 steveb8n I should have mentioned that these deps are all projects that I control but have split apart for re-use and/or separation of concerns. so I can be sure I know when changes are being introduced. I would never do this for an external dependency #2020-08-21 03:34:05 seancorfield Consider using :local/root maybe, if you have them all checked out locally? #2020-08-21 03:34:54 seancorfield You can always use a :dev alias to use :override-deps to replace the regular deps with your local ones. #2020-08-21 03:43:43 steveb8n agreed. I am using :local/root for these deps for localhost. the annoying commits are required when I push a lib which I depend on in N other projects and then have to update shas in all of them #2020-08-21 03:44:42 steveb8n ideally CI/test would use latest for all these deps. then when I cut a prod release, I would update the shas (once) for all the deps #2020-08-21 03:45:13 steveb8n but I can see that I could introduce human error later i.e. forget 1 sha update in the prod build. that would be bad #2020-08-21 03:45:32 steveb8n that would qualify as “silly” #2020-08-21 03:46:42 steveb8n this maintenance problem seems to be pushing me towards a mono-repo but I like keeping these things separate #2020-08-21 04:07:10 seancorfield Our 100K+ line codebase at work is in a monorepo, with 30+ subprojects, and we build about a dozen deployment artifacts from that... #2020-08-21 04:14:44 steveb8n it’s something I will think about. thanks #2020-08-21 06:39:45 borkdude @U0510KXTU I’m using git submodules to prevent endless commits updating git deps. Easy to update them all to latest using one git command. Works with all build tools. #2020-08-21 07:25:13 steveb8n thanks. I’ll look into that as well. I’ve never used them before but, if they add no complexity, I’ll give them a try #2020-08-21 07:26:03 borkdude @U0510KXTU I'm using them in babashka, I have 4 of them now. #2020-08-21 07:27:03 steveb8n cool. I’ll look at your repos when I dig deeper #2020-08-21 07:27:06 borkdude Not everyone likes them, I'm not entirely sure why, they work well for me. #2020-08-21 11:29:46 practicalli I would certainly keep specific library dependency versions the same for any CI pipeline that had any chance of pushing things into production.
Are you able to set up separate CI pipeline that uses a different alias to pull 'latest' versions of libraries?
I prefer using :local/root rather than mono-repo or sub-modules as they feel simpler for the way I work. Nothing wrong with those approaches though... #2020-08-21 11:31:53 borkdude local/root works, but not if the repo you're including is not part of the codebase proper, unless you're writing a script to set up the filesystem in CI the exact same way as on your machine #2020-08-21 11:34:05 practicalli Is that the case when using jars built from other projects, I thought that worked. Maybe I am mis-remembering. #2020-08-21 11:35:05 borkdude I mean, with local/root to some other project, you'll also have to git clone that other project in CI right #2020-08-21 11:36:54 practicalli Ah, yes. #2020-08-21 23:22:37 steveb8n I feel the same @U05254DQM in that I like them all separate. it has the added benefit of allowing me to outsource work on projects without needing to grant access to all #2020-08-21 23:23:36 steveb8n so for me it’s.. local deps using local/root, CI using git/sha and prod using the same git/sha #2020-08-21 23:24:35 steveb8n and the maintenance annoyance is updating the git shas for the CI builds anytime upstream projects change. #2020-08-21 23:25:32 steveb8n from this thread, I’ve realised that this is also a good thing because it means that my CI builds always match prod builds. thus it’s probably the lesser of all evils, despite the chattiness of commits #2020-08-24 08:15:33 rickmoynihan @U04V15CAJ: I’ve done the git submodule thing many times in the past. I’ve found it works quite well if you’re careful and have infrequently changing dependencies, but it can become pretty painful if you change branches a lot and have fast moving dependencies.
The detached heads in the submodules can also lead to people unwittingly making many kinds of mistakes. #2020-08-24 08:20:49 borkdude git pull --recurse-submodules #2020-08-24 08:21:08 borkdude I almost never have such mistakes, but maybe for contributors it's different? #2020-08-21 12:29:36 alexmiller Not building any new test runner but you can make tasks that do whatever #2020-08-21 10:34:45 otfrom and I'm struggling a bit with the deps.edn. Does anyone have a good example of how to reference a branch on github at a :git/url ? #2020-08-21 12:32:13 alexmiller Git deps always use shas so you don’t need to specify the branch #2020-08-21 12:32:27 alexmiller Sha is sufficient #2020-08-21 15:03:36 otfrom thx! I've got it working now. It was a problem with the deps.edn in the repo I was referencing. #2020-08-21 15:03:57 alexmiller ah #2020-08-21 10:41:36 otfrom I have it working locally in the mean time #2020-08-24 22:55:41 tolitius need to build a thin jar with one namespace aot compiled. the namespace has no dependencies:
{:paths ["src" "classes"]
:deps {}
:aliases {:jar {:extra-deps {seancorfield/depstar {:mvn/version "1.0.97"}}
:main-opts ["-m" "hf.depstar.jar" "yang.jar"]}}}
$ clj -e "(compile 'yang.java)"; clj -A:jar
it does work, but it also brings these along:
0 Mon Aug 24 18:45:40 EDT 2020 clojure/core/
0 Mon Aug 24 18:45:40 EDT 2020 clojure/core/specs/
932 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__40.class
1178 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__65$fn__71.class
798 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__46.class
929 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__27.class
936 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__101.class
929 Mon Aug 24 18:45:39 EDT 2020 clojure/core/specs/alpha$fn__25.class
...
I might be missing some args in depstar? #2020-08-24 22:58:36 seancorfield @tolitius Clojure core uses specs for language features (as macros), so those get dragged in because of that. #2020-08-24 22:59:12 tolitius thank you. is there a way to exclude these? #2020-08-24 22:59:28 tolitius when assembling a jar maybe #2020-08-24 22:59:36 seancorfield You'd have to manually remove them after your compile step right now. #2020-08-24 23:00:26 seancorfield Feel free to open an issue on GH with those repro steps and I'll see about adding an option to exclude stuff via regex. #2020-08-24 23:01:06 tolitius great, would that be a depstar repo? #2020-08-24 23:01:15 seancorfield seancorfield/depstar #2020-08-24 23:01:25 tolitius yep, will do. thank you Sean #2020-08-24 23:02:31 seancorfield There's already quite a few changes on develop so if I incorporate that exclusion this week, I'll cut a 1.1 release. #2020-08-24 23:03:09 tolitius oh.. nice. that’s fast #2020-08-24 23:07:00 seancorfield I'm off work Thu/Fri so I expect I'll do a round of OSS updates and releases before next Monday 🙂 #2020-08-24 23:25:49 tolitius in case someone sees this before the next monday:
clj -e "(compile 'yang.java)"; clj -A:jar; zip -d yang.jar clojure/\*;
does it #2020-08-24 23:53:51 seancorfield @tolitius If you don't mind depending on develop, you can test that right now. #2020-08-24 23:59:49 seancorfield Note that it excludes files not directories, so you'd still get the (empty) clojure/core/ and clojure/core/specs/ folders if you --exclude "clojure/.*" #2020-08-25 00:24:41 tolitius :main-opts ["-m" "hf.depstar.jar" "yang.jar" "--exclude" "clojure/core/specs/alpha.*"]}
756 Mon Aug 24 20:21:36 EDT 2020 yang/java$map__GT_edn$iter__130__134.class
3064 Mon Aug 24 20:21:36 EDT 2020 yang/java__init.class
1316 Mon Aug 24 20:21:36 EDT 2020 yang/java$fn__128.class
2003 Mon Aug 24 20:21:36 EDT 2020 yang/java$map__GT_edn$iter__130__134$fn__135$fn__136.class
0 Mon Aug 24 20:23:37 EDT 2020 clojure/
0 Mon Aug 24 20:23:37 EDT 2020 clojure/core/
0 Mon Aug 24 20:23:37 EDT 2020 clojure/core/specs/
0 Mon Aug 24 20:23:37 EDT 2020 tolitius/
1867 Mon Aug 24 20:21:36 EDT 2020 tolitius/Yang.class
yep, it works ) #2020-08-25 00:32:44 seancorfield Cool, thanks for testing! #2020-08-25 11:47:02 Janne Sauvala Can I specify version ranges in deps.edn for dependencies? #2020-08-25 12:08:18 vlaaad no #2020-08-25 17:04:58 seancorfield @janne.sauvala Yes, you can, but you shouldn't. The string used for the version in :mvn/version is just passed through to the underlying library, e.g.,
(! 1176)-> clj -Sdeps '{:deps {hiccup {:mvn/version "(,1)"}}}'
DEPRECATED: Libs must be qualified, change hiccup => hiccup/hiccup
Downloading: hiccup/hiccup/1.0.0-SNAPSHOT/maven-metadata.xml from clojars
Downloading: hiccup/hiccup/1.0.0-SNAPSHOT/hiccup-1.0.0-20120218.070543-1.pom from clojars
Downloading: hiccup/hiccup/1.0.0-SNAPSHOT/hiccup-1.0.0-20120218.070543-1.jar from clojars
Clojure 1.10.1
user=> #2020-08-25 17:06:24 seancorfield Current version is 1.0.5 but you can get the highest version below that
(! 1177)-> clj -Sdeps '{:deps {hiccup {:mvn/version "(,1.0.5)"}}}'
DEPRECATED: Libs must be qualified, change hiccup => hiccup/hiccup
Downloading: hiccup/hiccup/maven-metadata.xml from clojars
Downloading: hiccup/hiccup/1.0.4/hiccup-1.0.4.pom from clojars
Downloading: hiccup/hiccup/1.0.4/hiccup-1.0.4.jar from clojars
Clojure 1.10.1
user=>
But version ranges are "Bad(tm)" and should be avoided. #2020-08-25 17:06:45 seancorfield ^ @vlaaad FYI #2020-08-25 17:15:00 alexmiller it's not actually passed through, it's resolved in canonicalization to the max in range #2020-08-25 17:16:14 alexmiller but a) version ranges have many known weirdnesses and b) as a non-specific version, they also break clj's caching (you will retain the same cached classpath even if newer versions become available) #2020-08-25 17:16:21 Janne Sauvala Interesting to see it works, thanks @seancorfield! I realized I haven’t seen anyone using version ranges so I was wondering were those intentionally not supported #2020-08-25 17:16:31 alexmiller they are officially not supported #2020-08-25 17:16:40 alexmiller but unofficially handled because they exist in the wild, particularly with transitive deps #2020-08-25 17:18:20 seancorfield @alexmiller Good to know re: canonicalization -- I passed in a bad range and the exception came from Aether so I assumed it was just being passed through. My bad:
Unbounded version range (1,
org.eclipse.aether.resolution.VersionRangeResolutionException: Failed to resolve version range for hiccup:hiccup:jar:(1,: Unbounded version range (1,
at org.apache.maven.repository.internal.DefaultVersionRangeResolver.resolveVersionRange(DefaultVersionRangeResolver.java:137)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveVersionRange(DefaultRepositorySystem.java:247)
#2020-08-25 17:19:09 seancorfield > I haven’t seen anyone using version ranges
Because they're bad and folks avoid them 🙂 #2020-08-25 17:24:55 Janne Sauvala Yes. Too bad I have seen them used quite a lot in npm and python world. It is good that bad practice hasn’t polluted clojure community 🙂 #2020-08-25 17:19:56 alexmiller well the resolution happens via aether #2020-08-25 17:20:24 alexmiller I think if you had more of stack, you'd probably see canonicalize in there somewhere #2020-08-26 21:26:10 jhacks Trying out antq (https://github.com/liquidz/antq) and depot (https://github.com/Olical/depot) with these one-liners I get many hundreds of lines of DEBUG statements:
- clojure -Sdeps '{:deps {antq/antq {:mvn/version "RELEASE"}}}' -m antq.core
- clojure -Sdeps '{:deps {olical/depot {:mvn/version "2.0.1"}}}' -m depot.outdated.main
Is this normal or is there something up with my system? #2020-08-26 21:30:50 seancorfield @jhacks What do you mean by "hundreds of lines of DEBUG statements"? #2020-08-26 21:31:16 seancorfield I just tried antq.core on a project and got this:
> clojure -Sdeps '{:deps {antq/antq {:mvn/version "RELEASE"}}}' -m antq.core
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
| :file | :name | :version | :latest-version |
|----------+-----------------------------------------------------------------+------------+-----------------|
| deps.edn | com.cognitect/test-runner | | Failed to fetch |
| | com.h2database/h2 | 1.4.199 | 1.4.200 |
| | com.microsoft.sqlserver/mssql-jdbc | 8.2.1.jre8 | 8.4.0.jre14 |
| | com.zaxxer/HikariCP | 3.4.2 | 3.4.5 |
| | io.zonky.test.postgres/embedded-postgres-binaries-darwin-amd64 | 12.2.0 | 12.3.0 |
...
which seems perfectly reasonable. #2020-08-26 21:32:04 seancorfield depot produced this
> clojure -Sdeps '{:deps {olical/depot {:mvn/version "2.0.1"}}}' -m depot.outdated.main
Downloading: olical/depot/2.0.1/depot-2.0.1.pom from clojars
Downloading: olical/depot/2.0.1/depot-2.0.1.jar from clojars
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Checking for old versions in: deps.edn
org.clojure/java.data {:mvn/version "1.0.78"} -> {:mvn/version "1.0.86"} #2020-08-26 21:36:48 jhacks @seancorfield For this command clojure -Sdeps '{:deps {olical/depot {:mvn/version "2.0.1"}}}' -m depot.outdated.main
I get many lines like this before ending with the usual output:
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << Age: 7291592
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Served-By: cache-bwi5137-BWI, cache-wdc5525-WDC
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Cache: HIT, HIT
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Cache-Hits: 1, 1
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.headers - << X-Timer: S1598477735.173106,VS0,VE1
17:35:35.193 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.client.DefaultHttpClient - Connection can be kept alive indefinitely
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<html>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<head><title>404 Not Found</title></head>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<body bgcolor="white">[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<center><h1>404 Not Found</h1></center>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<hr><center>nginx</center>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "</body>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "</html>[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "<!-- a padding to disable MSIE and Chrome friendly error page -->[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.wire - << "[\n]"
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection [id: 49][route: {s}->] can be kept alive indefinitely
17:35:35.194 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection released: [id: 49][route: {s}->][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 100]
17:35:35.195 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:35:35.196 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51708<->199.232.64.209:443 closed
17:35:35.196 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51708<->199.232.64.209:443 closed
17:35:35.197 [clojure-agent-send-off-pool-2] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:35:35.198 [clojure-agent-send-off-pool-2] DEBUG org.eclipse.aether.internal.impl.TrackingFileManager - Writing tracking file /Users/joe/.m2/repository/selmer/selmer/resolver-status.properties
Checking for old versions in: deps.edn
cljs-ajax {:mvn/version "0.8.0"} -> {:mvn/version "0.8.1"}
com.google.javascript/closure-compiler-unshaded {:mvn/version "v20200504"} -> {:mvn/version "v20200719"}
day8.re-frame/http-fx {:mvn/version "0.1.6"} -> {:mvn/version "0.2.1"}
metosin/jsonista {:mvn/version "0.2.6"} -> {:mvn/version "0.2.7"}
org.clojure/core.async {:mvn/version "1.2.603"} -> {:mvn/version "1.3.610"}
org.postgresql/postgresql {:mvn/version "42.2.14"} -> {:mvn/version "42.2.16"}
re-frame {:mvn/version "1.0.0"} -> {:mvn/version "1.1.1"}
thheller/shadow-cljs {:mvn/version "2.10.17"} -> {:mvn/version "2.11.0"}
#2020-08-26 21:36:55 jhacks Sorry about the formatting #2020-08-26 21:40:45 jhacks For clojure -Sdeps '{:deps {antq/antq {:mvn/version "RELEASE"}}}' -m antq.core
I get lines like this:
17:38:47.822 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:38:47.822 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51766<->151.101.193.128:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51766<->151.101.193.128:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51765<->199.232.64.209:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51765<->199.232.64.209:443 closed
17:38:47.823 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager is shutting down
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51769<->199.232.64.209:443 closed
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.DefaultClientConnection - Connection 0.0.0.0:51769<->199.232.64.209:443 closed
17:38:47.825 [Finalizer] DEBUG org.apache.http.impl.conn.PoolingClientConnectionManager - Connection manager shut down
| :file | :name | :version | :latest-version |
|----------+-------------------------------------------------+-----------+-----------------|
| deps.edn | cider/piggieback | 0.5.0 | 0.5.1 |
| | cljs-ajax/cljs-ajax | 0.8.0 | 0.8.1 |
| | com.cognitect/test-runner | | Failed to fetch |
| | com.google.javascript/closure-compiler-unshaded | v20200504 | v20200719 |
| | day8.re-frame/http-fx | 0.1.6 | 0.2.1 |
| | metosin/jsonista | 0.2.6 | 0.2.7 |
| | org.clojure/core.async | 1.2.603 | 1.3.610 |
| | org.postgresql/postgresql | 42.2.14 | 42.2.16 |
| | re-frame/re-frame | 1.0.0 | 1.1.1 |
| | thheller/shadow-cljs | 2.10.17 | 2.11.0 |
#2020-08-26 21:47:14 jhacks I’m using clojure from homebrew and just updated it to see if that was causing it, but no change. There isn’t a debug setting somewhere with deps.edn? #2020-08-26 21:51:47 jhacks I tried it on another project that uses deps.edn (I only have two) and I don’t get the DEBUG output. It must be something specific to the project. Thanks for trying out the commands on your side @seancorfield! #2020-08-26 21:54:09 seancorfield Something in your project is enabling logging (at debug level) and those DEBUG messages are coming from the machinery that fetches metadata about artifact versions. #2020-08-26 21:55:59 seancorfield You may be able to resolve that by adding org.slf4j/slf4j-nop {:mvn/version "1.7.25"} to your command-line deps when running antq and/or depot @jhacks #2020-08-26 21:59:26 jhacks @seancorfield Wow! That got rid of the DEBUG statements on both of the commands. Thanks so much, that’s good to know about. #2020-08-26 22:00:33 seancorfield I use it with clj-new to suppress logging in project templates as it is generating them 🙂 #2020-08-26 22:05:02 jhacks Thanks for this video, btw: https://www.youtube.com/watch?v=CWjUccpFvrg I found it very helpful. I’ve been using lein and just getting started with deps.edn. clj-new is on my list to check out. 🙂 #2020-08-26 22:21:32 jhacks A lot of good info there 🙂 #2020-08-26 22:25:48 practicalli I learnt most things from Sean (and Alex too) #2020-08-28 12:10:43 rickmoynihan Are there any plans to support data-readers in deps.edn :args data? #2020-08-28 12:14:36 rickmoynihan The use case is defining overrides for an integrant system, allowing #ig/ref.
Obviously it’s also easy to put that system in a separate file, and read that, so not a big deal, and I can imagine that it’d add complexity to tools.deps. Just curious. #2020-08-28 12:43:32 alexmiller I don’t think there’s any need for that #2020-08-28 12:44:47 alexmiller Or do you mean for data in aliases in deps.edn? #2020-08-28 12:48:32 dominicm for context #ig/ref creates a record type. I suspect Rick is trying to pass in the integrant system to run via deps.edn aliases. #2020-08-28 13:04:20 alexmiller when you use -X the alias data is actually loaded in the context of the program you're running, so the data readers available via data_readers.clj[c] are available (but not currently being used in the edn/read, so seems like this shouldn't be too hard #2020-08-28 13:41:40 rickmoynihan Yeah it’s for data in aliases #2020-08-28 13:43:09 rickmoynihan And yeah I was thinking the same, using data_readers.clj(c) would be great for this. #2020-08-28 13:45:47 alexmiller I will log a jira for this, seems a reasonable thing to want #2020-08-28 13:46:43 rickmoynihan @dominicm’s suspicions are right… Essentially I’m writing a script, that wants to leverage some integrant components pre-defined in libraries. I currently have the system defined in a var in my program, and it would be useful to be able to override some of the components configs via args.
I could obviously just put it in a separate edn config file; but it’s another config file (sigh) for something that’s intended to be pretty minimal. #2020-08-28 13:47:05 alexmiller yep, got it #2020-08-28 13:48:21 rickmoynihan It would be fab if tools.deps could do this… it’d make configuring and overriding bits of integrant systems much nicer. #2020-08-28 13:49:36 rickmoynihan Though I warn you, it’ll be a slippery slope, I’ll probably next want data aliases and command line args to be meta-merged 😆 #2020-08-28 13:53:28 alexmiller what does "meta-merged" mean? #2020-08-28 14:16:39 rickmoynihan Well I was just joking about supporting meta-merge 🙂
But I meant:
https://github.com/weavejester/meta-merge
which is very useful for doing deep merges of data.
We tend to structure our integrant systems into profiles that get applied and meta-merged in a certain order. #2020-08-28 14:20:02 rickmoynihan So in a tools.deps meta-merge context you could have data aliases for each of those profiles, and meta-merge them all together in the specified order, with -Sdeps and command line args on top etc.
I was only joking though assoc-in makes more sense for tools.deps. #2020-08-28 14:20:30 alexmiller well, that's there for -X with an :args map and pathed data on the command line #2020-08-28 14:21:32 alexmiller I'm not going to use meta-merge on deps data, but your program can combine data from multiple aliases that you load in any way you want #2020-08-28 14:21:42 alexmiller by loading it via the runtime basis #2020-08-28 14:22:26 rickmoynihan 👍 Ahh yes that’s true — I’d forgotten I could do that. #2020-08-28 14:23:18 rickmoynihan I think that would work quite nicely actually 🤔 #2020-08-28 14:24:39 rickmoynihan I already have it slurped in my REPL 😁 #2020-08-28 14:24:43 rickmoynihan Thanks #2020-08-28 14:25:13 alexmiller I guess that code could use data readers too :) #2020-08-28 14:25:26 rickmoynihan Yeah was just going to ask about that #2020-08-28 14:25:44 rickmoynihan 👍 #2020-08-28 14:26:02 alexmiller edn/read-string takes an optional opts map where you can pass :readers #2020-08-28 14:27:17 alexmiller (def basis (edn/read-string {:readers *data-readers*} (-> (System/getProperty "clojure.basis") jio/file slurp)))
#2020-08-28 14:27:22 alexmiller something like that? #2020-08-28 14:28:06 rickmoynihan yeah that would work nicely with TDEPS-163 #2020-08-28 14:28:46 alexmiller there are other places that edn/read-string that you don't control, so those should do this too #2020-08-28 14:29:00 alexmiller inside tools.deps (deps reader) and the exec stub (command line) #2020-08-28 14:29:42 rickmoynihan Yeah I can’t do anything without TDEPS-163 also supporting the readers properly #2020-08-28 14:30:09 alexmiller I probably won't get to it today, but it shouldn't be a big deal #2020-08-28 14:32:13 rickmoynihan No rush, but it’ll be great to see this #2020-08-28 14:33:20 rickmoynihan Other use cases for us are that we like to represent URIs as data, not strings… and we have quite a lot of URIs in config data. #2020-08-28 13:55:18 alexmiller I'm guessing you mean deps.edn and -Sdeps alias data merged? #2020-08-28 13:57:51 alexmiller those will be merged now, but perhaps not in the way you want :) #2020-09-01 20:11:01 ghadi @seancorfield do you use anything to deploy artifacts to S3 private repos with deps.edn? #2020-09-01 20:11:24 ghadi anyone can answer, just tagging Sean because he has the biggest dotfile #2020-09-01 20:12:31 seancorfield Nope, sorry. We don't use S3 for repos. #2020-09-01 20:30:58 ghadi thanks #2020-09-02 07:39:05 rickmoynihan Hmm I should really think about open sourcing our s3 deployer #2020-09-03 06:29:54 vlaaad deps-deploy is so simple, I'm almost ready to just use aether #2020-09-03 06:31:02 vlaaad what's missing is an easy way to build pom.xml, currently a lot of build tooling complexity is either about updating or reading the pom #2020-09-03 06:31:15 vlaaad I hope tools-build has something in this area #2020-09-03 07:36:32 rickmoynihan Yeah our one (not yet opensourced) is actually almost identical to deps deploy (it just wraps pomegrante/aether too), except it also supports private s3 wagons. I’m almost tempted to just patch deps-deploy with the code to do that, rather than extract and open ours. #2020-09-03 07:51:18 dominicm I'd like to see thought go into credential management fwiw. I had a stab and discovered it's very tricky. #2020-09-03 08:35:37 rickmoynihan The other option is I guess to use cognitects s3 api to drive it; I think that hooks into all the possible AWS credential mechanisms #2020-09-03 08:36:11 rickmoynihan Though I think that would require you facading things for aether/pomegranate #2020-09-03 08:40:16 dominicm That's fine for s3, less so for clojars creds #2020-09-04 14:51:12 borkdude I'm thinking about a feature for clj-kondo where library authors can publish config specific to their library and clj-kondo can detect this somehow. So their artifacts should have some predictable and unique location for this config. Has something like this been done for other tools, so I could look at some prior work? #2020-09-04 14:53:56 alexmiller I've been thinking about exactly this lately for clj, but have not settled on an answer #2020-09-04 14:54:16 alexmiller one answer in the new clj world is to put it in :aliases of course #2020-09-04 14:54:43 alexmiller in general, we're trying to avoid "magic name" though #2020-09-04 14:54:44 borkdude yup but deps.edn itself isn't often part of a published artifact right #2020-09-04 14:55:08 alexmiller right - have been thinking about some way to allow tools to be self-describing #2020-09-04 15:11:47 borkdude I guess it doesn't have to be unique. If every library would publish their exported config under /clj-kondo, I guess you could just do .getResources. But maybe that doesn't work for dirs.
user=> (enumeration-seq (.getResources (.getContextClassLoader (Thread/currentThread)) "README.md"))
(#object[java.net.URL 0x50acf55d "jar:file:/Users/borkdude/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20170809-b9c14c6b/google-closure-library-third-party-0.0-20170809-b9c14c6b.jar!/README.md"] #object[java.net.URL 0x3cae7b8b "jar:file:/Users/borkdude/.m2/repository/org/clojure/google-closure-library/0.0-20170809-b9c14c6b/google-closure-library-0.0-20170809-b9c14c6b.jar!/README.md"])
#2020-09-04 15:13:25 borkdude I guess it does work:
user=> (enumeration-seq (.getResources (.getContextClassLoader (Thread/currentThread)) "clojure"))
(#object[java.net.URL 0x27a7ef08 "jar:file:/Users/borkdude/.m2/repository/org/clojure/data.json/0.2.6/data.json-0.2.6.jar!/clojure"] #object[java.net.URL 0x280e8a1a "jar:file:/Users/borkdude/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar!/clojure"] #object[java.net.URL 0x11e33bac "jar:file:/Users/borkdude/.m2/repository/org/clojure/core.async/0.6.532/core.async-0.6.532.jar!/clojure"] ...)
#2020-09-04 15:41:11 borkdude There is still nothing like $HOME or ~ in tools.deps, am I right? #2020-09-04 15:41:28 ghadi correct #2020-09-04 15:42:27 borkdude ok #2020-09-04 15:46:06 borkdude The reason I'm asking is, I'm considering sort of this for clj-kondo's config resources:
:config-paths ["../base/clj-kondo", "~/dev/clj-kondo", "~/.m2/repository/rum/rum/0.12.3/rum-0.12.3.jar!/clj-kondo"]
and maybe clojure could help create that path somehow using deps.edn. Still a bit fuzzy on this. #2020-09-04 15:49:59 borkdude I think the users would invoke clojure -Spath for their project and clj-kondo would then scan this classpath for clj-kondo top level dirs, spitting them out to some file which it uses for future reference #2020-09-04 15:50:22 borkdude but then they'd still would add their $HOME based paths manually, which is probably fine #2020-09-04 15:56:58 alexmiller not going to interpret ~ #2020-09-04 15:58:23 alexmiller I think you should still consider alias data more, since it has existing sources and merging #2020-09-04 15:58:37 alexmiller and that data could point to files #2020-09-04 16:00:23 borkdude Oh, the :config-paths will be part of .clj-kondo/config.edn, not of deps.edn. I was just thinking how clojure (or other tools) could help, since some of the config could come from libraries #2020-09-04 16:57:02 dominicm Do ns-aliases work like :alias/foo? #2020-09-04 16:57:38 alexmiller no, they are aliases like namespace aliases #2020-09-04 16:58:06 alexmiller it's a map of alias to namespace, and you can use the alias in place of the namespace in your function name #2020-09-04 16:58:43 dominicm Ohh, it's not for keys in the arguments. It's for the function name. #2020-09-04 16:58:53 dominicm I see now "symbol" in the reference, my mistake 🙂 #2020-09-04 16:59:17 alexmiller right. in your deps.edn {:aliases {:mine {:ns-aliases {foo my.other.thing}}}} and then clj -X:mine foo/bar to invoke my.other.thing/bar #2020-09-04 16:59:34 alexmiller and may be used for other things in the future #2020-09-04 17:02:06 dominicm Curiosity getting the better of me, why aren't aliases referenced like #ref or [:alias :mine] or something that would distinguish it as a value for reference? That would allow for :exec-args to reference aliases, or the CLI. #2020-09-04 17:04:41 alexmiller sorry, not sure I'm getting the use case #2020-09-04 17:05:04 alexmiller :exec-args can reference aliases, but it's up to your program to do that resolution (via the injected basis) #2020-09-04 17:06:03 alexmiller I have thought about some kind of automatic resolution and using a tagged literal for that might be a good idea, but not ready to commit to that yet #2020-09-04 17:58:15 dominicm This is pretty much what I'm asking about. I don't want my code to be dependent on aliases necessarily, then I have to wrap everything... It just seems like there's pitfalls. #2020-09-04 17:06:13 seancorfield @alexmiller I'm really surprised at the removal of -R and -O since those have been around since day one, as well as the removal of several -S options. I can imagine that breaking some people's scripts (it definitely breaks a couple of ours at work). And the impending removal of the ability of -A to run :main-opts is also going to break a bunch of our workflows at work. #2020-09-04 17:07:37 alexmiller interested in that feedback if others share it #2020-09-04 17:08:03 alexmiller some of those could be made to work again for some period of time if widely desired #2020-09-04 17:08:37 seancorfield We have a helper script that you pass a list of aliases into and it runs clojure -A:those:aliases in multiple directories -- now that script needs to know whether/when to use -M instead of -A based solely on whether the alias include :main-opts... which... I mean how would it know? #2020-09-04 17:08:45 alexmiller length of "period of time" tbd as well #2020-09-04 17:09:12 alexmiller can you take a look at frequency of that? #2020-09-04 17:09:26 seancorfield What do you mean "frequency of that"? #2020-09-04 17:09:56 alexmiller are 95% :main-opts or 50%? #2020-09-04 17:10:12 alexmiller or 5% #2020-09-04 17:11:13 seancorfield Pretty much any library/project out there built on deps.edn that includes instructions for running tests is going to suggest using -A:test or -A:test:runner and that would be broken if -A can't run :main-opts #2020-09-04 17:12:10 alexmiller yes, that's why it still works #2020-09-04 17:12:27 alexmiller but suggests to use -M instead #2020-09-04 17:13:19 alexmiller and only at some later time tbd will stop working #2020-09-04 17:14:01 alexmiller this is step 1 of a multi step plan towards a different place #2020-09-04 17:14:52 alexmiller but, also I'm trying to gauge the level of turmoil this will introduce :) #2020-09-04 17:15:27 seancorfield I think changing -A will introduce a lot of turmoil 🙂 #2020-09-04 17:18:16 alexmiller I await more feedback :) #2020-09-04 17:51:20 seancorfield @alexmiller I just checked, comparing behavior between stable (1.10.1.561) and prerelease (1.10.1.672) and there's no compatible way to specify arbitrary combinations of aliases that may or may not include :main-opts in a way that avoids the warnings from -A encountering :main-opts but still includes all the non-`:main-opts` deps. In other words, you can't provide a single command-line that works across stable/prerelease CLI versions the same way, without warnings, as far as I can tell? In stable, -M only brings in :main-opts so you either need -A or repeat the aliases with both -R and -M; in prerelease, if you use -A you get the warning and you can't use -R. Is that a correct reading of the expected behavior? #2020-09-04 17:52:09 borkdude What will be new the new equivalent for clojure -R:foo -C:foo -Spath? So getting the classpath with alias foo deps and paths included? #2020-09-04 17:52:56 seancorfield It seems like -M works as a -A replacement in prerelease but that can't be used on stable (because -M doesn't bring in general alias content in stable)? #2020-09-04 17:56:54 seancorfield @borkdude I don't think that's possible in the prerelease version? Since your only options are -A (which will still run :main-opts), -X, or -M... #2020-09-04 17:59:12 seancorfield Well, I think you can say clojure -Spath -X:foo but there's no command that is compatible across multiple clojure versions? #2020-09-04 17:59:14 borkdude right, but what if you need that? I know I'm using that now sometimes #2020-09-04 17:59:44 seancorfield (you can't say clojure -X:foo -Spath since it tries to interpret -Spath as a function to be executed) #2020-09-04 17:59:48 dominicm I use -R/C also when I have an alias with main opts, and dependencies and I only want the deps #2020-09-04 18:08:16 alexmiller if you want just parts of an alias, you should break that into multiple aliases and combine the parts you want #2020-09-04 18:08:49 alexmiller so make an alias for the deps, and an alias for main-opts, and use either one or both #2020-09-04 18:09:55 alexmiller from what I can tell, the number of people that use this very targeted usage of -R/-C is probably mostly the people in this conversation :) #2020-09-04 18:11:08 borkdude maybe we represent thousands of silent users struggling with this behind their corporate proxies ;) #2020-09-04 18:11:18 borkdude j/k #2020-09-04 18:11:26 seancorfield @alexmiller Am I correct that -M in stable does not include other stuff from the alias(es) but in prerelease it does include them? #2020-09-04 18:11:32 alexmiller yes #2020-09-04 18:12:01 seancorfield I was hoping there was a single command that could be used across multiple CLI versions (without warnings), but that seems not possible. #2020-09-04 18:12:36 alexmiller I don't think so. I want you to move to the new thing. :) #2020-09-04 18:12:59 borkdude so in summary: decomplect your aliases and move on? #2020-09-04 18:12:59 seancorfield Every project generated by clj-new so far is going to run into this (because of the :test:runner combination). #2020-09-04 18:14:24 alexmiller because -A:test:runner uses :main-opts and produces the warning? #2020-09-04 18:14:28 seancorfield Yup. #2020-09-04 18:14:48 seancorfield In particular, :runner adds the Cognitect runner and its :main-opts. #2020-09-04 18:14:50 alexmiller well the solution is to tell them to use -M:test:runner which is where I want people to end up #2020-09-04 18:15:08 seancorfield But -M:test:runner will not work on stable CLI today. #2020-09-04 18:15:13 alexmiller or even better to coax tools into providing -X friendly entry points #2020-09-04 18:15:41 alexmiller there may need to be a period of dual instructions #2020-09-04 18:17:22 alexmiller another option would be to leave -A alone, but basically deprecate it entirely and only doc other stuff going forward #2020-09-04 18:17:50 seancorfield Removing -R/`-C` completely means there is a non-zero amount of tooling that cannot work at all across multiple CLI versions. #2020-09-04 18:18:00 borkdude I think that would be the sanest: don't remove existing things unless you absolutely have to, undocument them and then wait for a year or so #2020-09-04 18:18:58 alexmiller there is a limit - leaving existing things complicates the impl. some of that's worth it, some isn't. #2020-09-04 18:19:20 alexmiller I think very very few people use -R and -C #2020-09-04 18:19:46 borkdude but a lot of people are using -A #2020-09-04 18:19:55 alexmiller yes #2020-09-04 18:20:06 alexmiller and it continues to work #2020-09-04 18:20:31 alexmiller with the suggestion to do something else :) #2020-09-04 18:20:50 borkdude printing a warning / hint is a good thing too imo #2020-09-04 18:20:55 alexmiller it does #2020-09-04 18:21:04 alexmiller you said above Sean "cannot work across multiple versions" but -A does #2020-09-04 18:22:14 borkdude At my job they are just starting to adopt deps.edn. I was away for 1.5 week and my colleague starting porting other projects to deps.edn, after I did only our front-end. He's even using -X. I told him that's only release candidate material ;) #2020-09-04 18:22:41 alexmiller he's a man of the future #2020-09-04 18:22:57 borkdude He also used your boot deps edn thing @seancorfield, since we still rely on boot for most of the stuff, but slowly migrating away #2020-09-04 18:23:15 alexmiller I'm hoping it will soon be in the stable version #2020-09-04 18:23:16 borkdude so I hope the breaking changes won't affect their initial impression #2020-09-04 18:24:18 borkdude but if warnings are printed, I think it's fine #2020-09-04 18:25:04 seancorfield @alexmiller So what is the recommended replacement for clojure -R:foo -C:foo -Spath ? #2020-09-04 18:25:37 borkdude Sean: he already told: break foo up into multiple aliases and use -A #2020-09-04 18:26:28 seancorfield So this is accepted as a breaking change? That will force "lots" of users of the CLI to modify their deps.edn files? #2020-09-04 18:26:38 alexmiller clj -Spath -M:foo -P #2020-09-04 18:27:07 seancorfield Because -P suppresses :main-opts? #2020-09-04 18:27:14 alexmiller it suppresses execution #2020-09-04 18:27:59 borkdude that's also slightly confusing to me, always has been, like -A:foo -Spath.... is it going to execute main or not... I always have to try it since I can't remember #2020-09-04 18:28:00 alexmiller it may be that -Spath should imply -P, I haven't thought about that much #2020-09-04 18:28:15 seancorfield But there's no compatible equivalent that works across both stable and prerelease? #2020-09-04 18:28:18 alexmiller I will make a note to look at that next week #2020-09-04 18:28:41 alexmiller like I said, hadn't thought about it #2020-09-04 18:28:50 Derek Passen What’s the reasoning for -Spath to continue to exist but -Stree to move to -X:deps tree? #2020-09-04 18:29:08 seancorfield Because -Spath is handled in the script and doesn't need to run a program. #2020-09-04 18:29:21 Derek Passen Thank you #2020-09-04 18:29:31 borkdude that's not entirely true though? -Spath uses the tools jar which is a JVM program #2020-09-04 18:30:25 borkdude if you have a cache, then it's indeed script only, I get the difference now #2020-09-04 18:30:27 seancorfield Only if it isn't cached -- it's related to just the first "part" of the CLI: build classpath etc. #2020-09-04 18:30:47 alexmiller yeah, what Sean said #2020-09-04 18:31:05 alexmiller it seemed silly to pass a path to a Clojure program so it could print it :) #2020-09-04 18:31:36 alexmiller we originally had it in the list of things to change but just didn't make sense (-Sdescribe is similar) #2020-09-04 18:31:47 seancorfield I think my biggest concern here is that several things that work in stable will simply break in the next version. #2020-09-04 18:31:47 borkdude I view clojure as a tool for creating classpaths, and incidentally also executing them ;) #2020-09-04 18:32:02 alexmiller yes, there are breaking changes #2020-09-04 18:32:59 seancorfield Which means that if you use CLI tooling across a team and across various servers/CI/etc, you have to have a coordinated upgrade of the scripts, JARs, and all the changes to your code/tooling. #2020-09-04 18:33:29 alexmiller I've spent a week deciding which migration strategy to take on each of these settings. I tried to make what I judged to be the common things continue to work #2020-09-04 18:34:15 alexmiller none of these is a final decision, that's why it's out here as a prerelease for feedback #2020-09-04 18:34:43 alexmiller the only thing I've heard that is actually breaking you now is the -Spom I think? #2020-09-04 18:35:06 alexmiller well let me flip it and say, please give me a list of what is breaking you #2020-09-04 18:36:10 seancorfield It breaks our "outdated dependency" script, which relies on -Stree and now there's no compatible way to run that script across multiple versions of the CLI. #2020-09-04 18:37:58 alexmiller making -Stree or -Spom continue to work is an easy thing to do #2020-09-04 18:38:55 alexmiller (by internally rewriting as -X) #2020-09-04 18:39:23 seancorfield We don't use -R/`-C` but that will be a breakage for others with no compatible way to do it across multiple versions of the CLI. #2020-09-04 18:39:42 alexmiller I don't believe there are more than a handful of people doing that #2020-09-04 18:39:58 alexmiller (but I will be watching to see if I'm wrong) #2020-09-04 18:40:47 borkdude False if old, true if new, just dropping this here :)
$ clojure -Sdescribe | bb '(-> *input* :version (str/split #"\.") (->> (mapv #(Integer. %))) (compare [1 10 1 672]) nat-int?)'
false
#2020-09-04 18:41:41 alexmiller what am I reading? #2020-09-04 18:41:53 seancorfield CLI version sniffing 🙂 #2020-09-04 18:42:01 alexmiller oh, ok #2020-09-04 18:44:48 alexmiller @seancorfield do you use -Stree with other args like -A/R/C? #2020-09-04 18:48:27 seancorfield We use it with -A #2020-09-04 18:50:01 seancorfield Currently, we use -A everywhere in our scripts that drive CLI stuff, because we have :main-opts in "separate" aliases -- but those :main-opts are mostly accompanied by :extra-deps to bring in deps for those mains. #2020-09-04 18:50:59 seancorfield The latter means we can't switch to anything that portably runs across multiple CLI versions that doesn't also produce warnings that might interfere with any output parsing we might do. #2020-09-04 18:52:23 borkdude What about a CLJ_VERSION variable that invokes the right version of your CLI tool? #2020-09-04 18:52:54 alexmiller if the warnings were all stderr, would you not care? #2020-09-04 18:53:05 alexmiller (right now they are not all stderr, but they could be) #2020-09-04 18:53:27 seancorfield Well, at this point I'm now considering putting clojure and the tools JAR into our repo under version control so we can guarantee a given version everywhere, even if a developer or server environment has a different version of the tooling... #2020-09-04 18:57:12 alexmiller fyi, there are two jars now :) #2020-09-04 18:57:59 alexmiller -X uses a bootstrap program that does not have any other dependencies and that is in its own jar as of this release #2020-09-04 18:58:42 seancorfield Yeah, a couple of our servers are old enough that I can no longer run the CLI install shell script on them so any upgrades I do on those servers is manual and that was a bit painful because there's more than just a single script to update. #2020-09-04 18:58:59 seancorfield lein/`boot` both used wrappers that auto-downloaded the JARs if they were missing. #2020-09-04 18:59:36 seancorfield But making that work reliably across every environment, including older envs, is tough -- and not good for controlled corp envs with firewall restrictions etc. #2020-09-04 19:00:35 seancorfield And moving this into our repo means we can also move all our tooling to leverage the new features faster without worrying about other devs or envs having older versions of the CLI stuff. So that would be a win, I guess. #2020-09-04 19:01:10 borkdude @seancorfield I'm doing the same with deps.clj (downloading the jar that is). It also has support for proxies. #2020-09-04 19:02:35 borkdude I'm considering adding download instructions when it fails for some reason. I'm not sure if I'm allowed to bundle those jars, so I'm doing it that way mostly for that reason #2020-09-04 18:53:52 seancorfield That's never been a problem before because there have not been any outright breaking changes. #2020-09-04 18:54:03 borkdude I'm willing to consider something for deps.clj which allows compatibility with both versions #2020-09-04 18:54:12 borkdude based on some env var #2020-09-04 18:56:11 seancorfield (we used to keep the lein and boot scripts under version control to avoid these sorts of problems) #2020-09-04 18:56:46 borkdude I guess that also works #2020-09-04 19:03:34 alexmiller I'm taking off - have a good weekend all. more feedback is fine, just won't see it today :) #2020-09-04 19:03:46 borkdude Have a good weekend! #2020-09-04 19:05:46 seancorfield The lack of "portable" CLI commands/instructions across multiple (unknown) versions of the tooling is definitely my biggest concern about this... #2020-09-04 19:06:59 borkdude Is the issue that the alphabet is too short to use different letters? What about longer option names? #2020-09-04 19:08:00 seancorfield I don't think that solves anything @borkdude -- -M is the natural naming to choose here. It's just the transition between versions that is going to be so painful. #2020-09-04 19:09:57 borkdude I guess especially on CI where you can't interactively react on a warning, yes #2020-09-04 19:10:44 borkdude Checking in the CLI script is a good idea, probably we should also adopt it. #2020-09-04 19:15:33 seancorfield Since clj-new generates new projects for people and those include instructions on how to run tests etc, I'll have to "dual instruction" all of those... #2020-09-04 19:16:51 seancorfield Old model: -A:test:runner
New model: -M:test:runner
New command doesn't work on old CLI (because -M doesn't pull in :extra-deps) and old command produces a warning on new CLI and users -- especially beginners -- will complain that clj-new is "doing something wrong"... #2020-09-04 19:35:48 seancorfield clojure -A:test:runner -Spath works on both stable and prerelease because the -Spath option prevents execution -- but the prerelease produces a warning about switching to -M, but if you do what the warning says: clojure -M:test:runner -Spath treats -Spath as an argument for clojure.main #2020-09-08 15:44:51 alexmiller I didn't see this go by before but order matters here - the exec opt should be last (same thing you mention later with -X) because in both cases you are passing arbitrary args to another program #2020-09-08 15:45:04 alexmiller I understand why that's particularly confusing in this example #2020-09-08 16:53:15 seancorfield clojure -Spath -A:test:runner -- works on stable; works on prerelease but produces a warning (use -M)
clojure -Spath -M:test:runner -- works on prerelease (so switching -A to -M per the warning behaves "as expected")
clojure -A:test:runner -Spath -- works on stable; works on prerelease but produces a warning (use -M)
clojure -M:test:runner -Spath -- does not work on prerelease (so switching -A to -M per the warning "stops working")
It's probably not going to be obvious to a user why prerelease is more sensitive to order than stable. #2020-09-08 16:53:21 seancorfield If -Spath implied -P, I think that would provide fewer surprises. #2020-09-08 17:26:10 alexmiller Yeah #2020-09-08 17:26:18 alexmiller I took a note of that #2020-09-09 19:41:28 alexmiller have thought about this more. -Spath does already have the same effect as -P, I think the issues here are the warnings (I don't actually think those are an issue) and the ordering constraint. I could probably detect the -M case but I'm not sure it's actually worth doing so. #2020-09-09 19:43:23 seancorfield Maybe update the docs around -Spath to make it clear that it should be before -M or -A on the command-line? #2020-09-09 19:45:55 alexmiller the ordering is explicit in the docs etc already #2020-09-09 19:46:29 alexmiller clj-opt is always listed first and exec-opts listed last #2020-09-09 19:47:27 alexmiller might be someplace it could be made clearer #2020-09-09 19:47:58 seancorfield Ah, I had certainly missed that subtlety... #2020-09-09 19:49:30 alexmiller I'll review next time I update docs #2020-09-09 19:49:41 seancorfield I suspect that I'd gotten used to it working on stable with -Spath at the end #2020-09-09 19:51:33 seancorfield (and, indeed, everywhere I've ever used -Spath it's at the end -- in all my CircleCI configurations and test runner scripts, and even in READMEs where I'm illustrating a usage of -Spath!) #2020-09-04 19:37:11 seancorfield $ clojure -Sforce -A:test:runner -Spath
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
/Users/sean/.m2/repository/org/apache...
so you think OK,
$ clojure -Sforce -M:test:runner -Spath
Unknown option: "-S"
Unknown option: "-p"
Unknown option: "-a"
Unknown option: "-t"
Unknown option: "-h"
USAGE:
clj -m cognitect.test-runner <options>
🙂 #2020-09-04 19:41:27 seancorfield (don't get me wrong: I think the changes overall are good and the direction is good -- it's just the breakage and potential confusion that I think is going to be very problematic) #2020-09-04 21:44:37 lread My 2 cents, fwiw, is that breakage is totally acceptable for an alpha release. Since so many folks are relying on this alpha release component, what would be nice, where possible, is clear guidance. For example I just tried the following with v1.10.1.672:
> clojure -Spom
Option changed, use: clj -X:deps mvn-pom
My usage broke, but the fix is ultra clear.
Also since most folks just install Clojure latest with their package managers, it might be nice to highlight how to stick with the previous release until they can schedule migration. This might not help with CI tools that install the latest automatically, so a good amount of forewarning would likely be appreciated so that CI scripts can be updated.
And even though you have my 2 cents, there are sure to be folks who don’t follow Clojure closely, and don’t fully realize they have been using a changing alpha component (they knew they were using 1.10.1 but didn’t really realize they were using 1.10.1.xxx) and will be surprised because Clojure is very much known and valued for not breaking on update. #2020-09-04 22:19:28 seancorfield @lee I agree that tools.deps.alpha itself is clearly alpha and subject to change, but I think a lot of people would read https://clojure.org/guides/getting_started and not consider the CLI itself to be alpha and therefore not expect it to "randomly break stuff". #2020-09-04 22:22:45 seancorfield And if two different beginners have installed the CLI at different times (one before this change and one after this change), their experience following instructions out there on the web are going to be confusingly different. #2020-09-04 22:24:59 seancorfield This will be compounded if they try to follow tutorials out there about the CLI and create new projects (with clj-new), they'll also have confusingly different experiences... #2020-09-04 22:27:17 seancorfield I sympathize that the path from where we are today (at least on the stable CLI) to where we want to be "tomorrow" (when the prerelease becomes stable.next) is complicated by retaining backward compatibility here... #2020-09-04 22:30:07 dominicm Unrelated, is it/will it be considered to specify extra-deps via an alias map? Between that & paths, it would abate most complaints about copy/paste between aliases / remembering which combination of aliases to compose together.
Clasically it's :test:run because :run-tests requires copying the paths/deps from :test and people aren't so keen on that.
Not something I'm looking to drive, but curious about how that would impact future endeavours of mine. #2020-09-04 22:32:19 seancorfield I think leaving -A alone -- retaining exactly its stable behavior, without even printing a warning message -- and just removing it from the documentation might be the best path. And perhaps at some future date adding a printed warning about using an undocumented option. That will allow a lot of tutorials etc to continue working -- without any potentially confusing warnings -- and let new/updated tutorials start talking the new options (but with caveats about CLI versions, to cater for folks who have older installations). #2020-09-04 22:33:27 lread @seancorfield and @borkdude I too empathize with folks who will be surprised by any cli breakages. And agree that it is likely not clear to the average Clojurian that they have been using an alpha component. #2020-09-04 22:35:36 seancorfield @dominicm I already have :test:runner so I can use :test in various contexts without needing Cognitect's test runner, but today -A:test -M:runner doesn't work with that setup (since -M today does not bring in :extra-deps). So either have to split :runner in two and use something like -A:test:runner -M:run-tests to avoid warnings and be compatible across both versions or provide dual instructions that are version specific (`-A:test:runner` today, -M:test:runner "tomorrow"). #2020-09-04 23:03:52 nate Just another vote for what @seancorfield is saying. I use -C for mixing in a classpath into a -A invocation. I see how this can be changed for the next stable version, but having a window of deprecation would really be nice. #2020-09-05 06:42:14 seancorfield Reading over the prerelease Deps/CLI reference again, and reviewing the discussions above, it seems the -A option is being promoted as the way to do everything-except-execution whereas before it used to include execution. Given that it's being referred to as the "REPL option" and, for the most part, -R used to be somewhat similar to -A without :main-opts, I wonder if keeping -R but making it also pull in the classpath stuff (but not :main-opts) would be the less disruptive path? #2020-09-05 06:42:38 seancorfield Then -A could quietly be undocumented, eventually deprecated, and finally removed. #2020-09-05 06:44:33 seancorfield This would be a minor expansion to what -R currently does (it would respect :extra-paths which only -A/`-C` did before) and it would still ignore :main-opts. I suspect -C is the least used option and almost all -A uses could be replaced by either -M (in the new world) or -R (in either world) with no changes in behavior. #2020-09-05 06:45:09 seancorfield And -R more closely relates to REPL. So we'd have -R/REPL, -X/execute, and -M/main. #2020-09-05 06:50:24 seancorfield Hmm, I guess -R would also have to subsume -O for that to work fully. #2020-09-05 06:53:12 seancorfield Going back to the :test:runner example, it seems like -A:test:runner -M:runner works identically on both stable and prerelease and produces no warning on prerelease (presumably because -M is present?). #2020-09-05 06:54:14 seancorfield Ugh! So... I guess instructions could change now to recommend -A... -M... and that would work now and tomorrow even though it's a bit redundant. #2020-09-05 07:05:43 borkdude @seancorfield I like your suggestion of extending R to include C, I've always found it weird that these two were separated. Also the -A -M trick is something I tried yesterday, but somehow didn't work for me, since I got file not found -M, but now that I'm trying it again, it seems to work. #2020-09-05 07:08:04 seancorfield Yeah, having integrated CLI/t.d.a into our repo and started to update all our script to match this "new world", I'm very torn between "extending -R and undocumenting -A" and just "documenting -A... -M..." as the preferred approach. #2020-09-05 07:09:16 seancorfield I would much prefer a solution that works for today and tomorrow (which is the -A... -M... approach, especially since that suppresses the warning on -A). #2020-09-05 07:10:32 seancorfield Either way, our build script that previously just turned build aliases subproject options into cd subproject; CLJ_CONFIG=../versions clojure -A:aliases options needs to be way smarter about handling the options part... #2020-09-05 07:10:39 borkdude Yeah, I guess clojure could override certain deprecated options when the more preferred ones are used, which provides a migration path #2020-09-05 07:10:54 seancorfield Well, it already does. #2020-09-05 07:11:27 borkdude Yes, but also for other deprecated options? #2020-09-05 07:11:32 seancorfield If you use -A:test:runner -M:runner you don't get the warning, even when :runner includes both :extra-deps, :extra-paths, and :main-opts. #2020-09-05 07:11:38 seancorfield Which deprecated options? #2020-09-05 07:11:46 borkdude -R, etc #2020-09-05 07:11:57 seancorfield -R, -T, and -O aren't deprecated, they're simply removed. #2020-09-05 07:12:12 seancorfield -A's use of :main-opts is deprecated. #2020-09-05 07:12:51 borkdude I've never used -T or -O, so R is what I'm concerned about. So if you use -R ... -A ... in the new world, does that do the same as in the old world? #2020-09-05 07:13:11 seancorfield -R cannot be used in the new world. It does not exist. #2020-09-05 07:13:19 borkdude but is it ignored? #2020-09-05 07:13:24 borkdude as in, no error #2020-09-05 07:13:32 seancorfield It spits out an error and halts the script. #2020-09-05 07:13:42 borkdude that's what I meant #2020-09-05 07:14:13 seancorfield (! 2172)-> clojure -R:test -A:test
-R is no longer supported, use -A for repl, -M for main, or -X for exec
Sat Sep 05 00:13:57
(sean)-(jobs:0)-(/Developer/workspace/wsmain/build)
(! 2173)-> echo $?
1
#2020-09-05 07:14:22 borkdude but I guess -A would also override -R in the old world, so in the old case it also doesn't work as a polyfill for both #2020-09-05 07:14:53 seancorfield Right, the problem is that -A in the old world also executes #2020-09-05 07:15:28 seancorfield But at least changing -A:... to -A... -M... works identically and doesn't issue a warning in either world. #2020-09-05 07:15:37 borkdude that's already a win #2020-09-05 07:15:56 seancorfield But anything that uses -R today is plain ol' broken tomorrow. #2020-09-05 07:16:49 seancorfield There's no compatible way to run anything that uses -R today in a way that works regardless of clojure version I think. #2020-09-05 07:17:30 seancorfield So you pin to an old version? #2020-09-05 07:17:53 borkdude I never thought about it really hard, just copied the install instructions and that seems to contain a version number #2020-09-05 07:18:08 seancorfield Fine for CI for you, but not fine for any library code or tools that have to live in a version where members could have any version of CLI installed... #2020-09-05 07:18:57 borkdude But in that arena people see warnings and can react to them #2020-09-05 07:19:46 borkdude which can still be confusing, no argument there #2020-09-05 07:20:35 seancorfield I have a solution for clj-new now so I'm somewhat mollified 🙂 but it doesn't help anything that uses -R and has to live in a multi-version world. #2020-09-05 07:21:23 borkdude I guess clojure could print a link to a more elaborate blog post about the situation #2020-09-05 07:21:59 seancorfield I don't consider that a solution. "We broke your shit. Go read this to fix it!" #2020-09-05 07:22:49 seancorfield Seriously, that is not the right approach, esp. given Rich's whole pulpit-bashing about no breaking changes without renaming things. #2020-09-05 07:24:01 borkdude I agree that it's far from ideal, but assuming that it's going to be like it is now, it provides people a place to read up if they are curious about why it changed. #2020-09-05 07:25:58 seancorfield Considering how many people learn Clojure by using lein and still trip over ~/.lein/profiles.clj stuff, I don't think we should just throw our hands up and accept this and just say "Well, read these docs about why we broke stuff". I really don't. #2020-09-05 07:27:02 borkdude You don't have to convince me that avoiding breakage would be preferred. I think I'm going to take a break from this channel for the rest of the day :) #2020-09-05 07:27:43 seancorfield I can change all the -A uses in clj-new's docs and in the generated projects to be -A... -M... and at least I then say, "Hey, the latest version does recommend using both options so you're future-proofed" but it's still a fairly sucky experience for users. #2020-09-05 07:28:07 seancorfield Enjoy your weekend @borkdude 🙂 #2020-09-05 07:44:03 seancorfield Proposal:
* Recommend authors change -A:stuff to -A:stuff -M:stuff for compatibility with both versions where they want :main-opts executed.
* Keep -R and expand it to include :extra-paths and :jvm-opts so it mnemonically matches REPL: -R/REPL, -X/eXecute, -M/Main.
* Undocument -A for now, officially deprecate it later (and add that warning about using -M instead), and eventually drop it altogether. #2020-09-05 14:25:22 alexmiller I think this is pretty good. I'll have to go back through my spreadsheets to evaluate more. I think the first bullet recommendation could even be for tool authors to do nothing, wait until most people are using a new version, then change their recommendation. #2020-09-05 14:30:11 seancorfield If you don't introduce the warning for -A -- per bullet 3 -- then, yes, bullet 1 isn't really necessary. #2020-09-05 14:32:28 alexmiller the warning is what tells people how to move to the new world - without that, won't people be stuck on -A forever? #2020-09-05 14:33:42 seancorfield Bullet 3 suggests introducing the warning some time later, after everyone has had a chance to update. #2020-09-05 14:35:00 seancorfield People don't like warnings from their day-to-day development tools. Beginners in particular will complain to library authors about "incorrect instructions" if you introduce that warning before anyone has had a chance to update instructions/tutorials. #2020-09-05 14:36:31 alexmiller I'm ok with that :) I'm not sure I'm in agreement with you on the scope of this as an issue #2020-09-05 14:37:02 alexmiller you're right that they don't like warnings, so they'll change what they do, which is what we want #2020-09-05 14:38:07 alexmiller library authors can either do nothing and situation is same
or they can use dual instructions for a time
or they can doc new instructions and say clj > X.Y.Z (which also gets people on new version) #2020-09-05 14:38:37 alexmiller or they can start using -X style invocation (which also drives to new version) #2020-09-05 14:39:04 seancorfield > I'm not sure I'm in agreement with you on the scope
I'm pretty sure we're just not in agreement on that 🙂 #2020-09-05 14:39:43 alexmiller there is a middle ground - no warning for -A on first stable release #2020-09-05 14:39:45 alexmiller then warning #2020-09-05 14:39:52 alexmiller then (much later?) removal #2020-09-05 14:41:02 seancorfield Having been on the receiving end of complaints about incorrect instructions (or version-specific instructions that weren't absolutely clear on how to determine the version and how to "solve" the problem, on all platforms), I'm perhaps more sensitive to this whole issue than you... 🙂 #2020-09-05 14:41:21 seancorfield Yes, bullet 3 is that middle ground. #2020-09-05 14:41:29 alexmiller I'm on the receiving end of those complaints too #2020-09-05 07:45:29 seancorfield This provides a migration path for the most common use cases that is minimally disruptive (most uses of -R today won't care about paths or JVM options I suspect). It improves naming for the three desired use cases: REPL, eXecution, and Main opts. It breaks the least amount of code out there. #2020-09-05 07:50:07 seancorfield (question for Alex when he gets back: why are there two identical copies of exec.jar in the CLI distribution -- one at the top level, one in libexec?) #2020-09-05 14:31:14 alexmiller I think there aren't two in the distribution, the install script copies the root one to libexec which is the structure brew expects, although I think maybe I didn't sync the clojure script up to that and it's using the root one. I'll check on that #2020-09-05 16:03:16 seancorfield FYI, confirmed that script uses the root one:
-classpath "$cp:$install_dir/exec.jar"
#2020-09-05 14:28:14 alexmiller Keep in mind that most Clojure users are on Mac and install via brew and that brew aggressively updates you to latest version (our stable) even if you're upgrading something else. So within some period of time (couple months maybe), many people are using newest version. #2020-09-05 14:28:14 alexmiller Keep in mind that most Clojure users are on Mac and install via brew and that brew aggressively updates you to latest version (our stable) even if you're upgrading something else. So within some period of time (couple months maybe), many people are using newest version. #2020-09-05 15:27:08 practicalli That only covers 55.36% of the survey responders, so this doesnt seem to be the strongest of arguments.
Of course I am happy to help encourage upgrading to newer versions in any way I can, once I've understood the changes myself. #2020-09-05 15:31:52 seancorfield I wonder what percentage of the 35%-ish that are on Linux are using brew vs manually installing via the .sh file? #2020-09-05 15:36:20 practicalli There are legion of package managers for Linux that brew competes against, maybe a question for the next survey?
I find it easier to use the Linux script install. #2020-09-05 15:51:45 seancorfield I use brew on Ubuntu (on WSL) because it's easy to keep clojure up-to-date that way but that's recent for me -- I used to use the manual install script, but I tended to only upgrade when I remembered to do it, or I tried to do something that my older, installed version didn't support 😐 #2020-09-05 21:18:11 dominicm I've used Linux for many years, the brew stuff is extremely rare. #2020-09-05 21:24:14 seancorfield Yup, I'm just glad it exists for clojure. Getting stuff into the more standard installer repos on Linux seems very laborious? What relatively painless options exist for Linux package managers that the Clojure core team could leverage @U09LZR36F? #2020-09-05 21:25:59 dominicm Ha! 😂 #2020-09-05 21:26:09 dominicm If only you knew the rabbit hole you'd just asked about #2020-09-05 21:27:15 dominicm Every distribution has its own rules and timelines.
Consider Ubuntu LTS. They don't take changes. Canonical pay people to backport security patches. Every now and then you get a system upgrade (like mac) and that's when you get changes. New features, bug fixes, etc. #2020-09-05 21:28:36 dominicm There are, of course cutting edge distributions like arch and void. They'll update whenever the package maintainer feels like it. Usually regularly for important changes. #2020-09-05 21:29:06 dominicm Some distributions care about reproducibility. They have other constraints I don't understand. #2020-09-05 21:30:24 dominicm I am a void clojure package contributor, and my life is pretty easy. The installer script was "ported" to the void build system in ~10 lines and it's just worked since then.
Maybe I'll need to make changes with the new system. #2020-09-06 00:02:52 practicalli @seancorfield To package Clojure for Ubuntu, then I'd suggest a personal package archive (PPA), new release of Clojue can be pushed to this as soon as they are ready. There is a specific process for creating package to learn. Packages can be built for each release of Ubuntu (e.g. current long term support and intermediate releases) and published on the PPA. The relevant packages can be submitted for each new Ubuntu release (6 month cycle).
There are also other options such as snaps and flatpack, but a proper *.deb package is much nicer in my opinion as it ensures there are no conflicts with other software installed via the package manager. There is a Leiningen package for Ubuntu, so some of the experience from that can be used.
For Arch, I seem to remember that @U11EL3P9U packages up clj-kondo , so I assume his work can be of help for packing to the Arch User Repository (AUR). #2020-09-06 05:43:53 dharrigan That's right, I do maintain a few Clojure packages on Arch, like clj-kondo, babashka, clojure-lsp. A pleasure to do so 🙂 However, the main Clojure package is part of the Community Repo that is under a different maintainership. It is reguarly updated (last update was 2020-08-26, which brought in Clojure 1.10.1.645). #2020-09-05 14:28:29 alexmiller Also keep in mind that we are in work on a 1.10.2 Clojure cycle and we tie brew versions to Clojure versions, so that might be a natural point to either coax an upgrade or drop some kind of compatibility #2020-09-05 15:56:59 seancorfield It works quite nicely that way... #2020-09-05 15:59:22 seancorfield I expect I'll update clj-new next, to support -X usage... #2020-09-05 16:34:28 borkdude > Keep in mind that most Clojure users are on Mac and install via brew
Have you considered that those Mac users will probably also use the linux scripts on CI? Do you get any stats from that usage? It may be less of an issue since the install script is versioned, but since people tend to develop CI scripts locally (I often do), it might become confusing. #2020-09-05 16:36:12 borkdude There might also be less Windows users using this because for some people it's harder to get going than lein. I got this report yesterday and it's not the first time I hear these kinds of issues.
> Clojure until 1.6 used to work on Windows from https://chocolatey.org/packages/clojure too, without problems. Clojure Tools broke everything and it seems that OSX ( Homebew ) is the only "first class citizen" now.
https://github.com/babashka/babashka.process/issues/15#issuecomment-687035007
Don't shoot the messenger, I don't really have a stake in Windows support because I know how to work around it :) #2020-09-05 20:32:48 alexmiller Oh I definitely think that’s a factor #2020-09-05 16:49:13 seancorfield Yeah, you have to be pretty persistent to use Clojure CLI/`deps.edn` on Windows, and willing to work with very "alpha" tooling. Which a lot of Windows users are not willing to do: they often have different expectations of the user experience than devs on macOS/Linux. #2020-09-05 16:51:27 borkdude I do use clojure on the other two major OS CIs #2020-09-05 16:53:57 seancorfield I'll be using Windows for development a lot more going forward (Microsoft had a great deal on Surface Laptop 3 so I just ordered an i7 with 16GB RAM and 256GB SSD). But I'll be leaning on WSL2 very heavily, and using (Linux) brew for installation. #2020-09-05 17:03:20 Quest I have an odd case -- at least from my pure Clojure background (didn't start in Java.)
I have a Java dependency that doesn't publish to maven. I can run ./gradlew build on it to get a library .jar.
What's the best way to include this into deps.edn dependencies? #2020-09-05 17:06:31 Quest Can I publish just a .jar to maven under my own repo?
I could try updating the library to properly publish to Maven & PR'ing it, but I'm just unfamiliar with the gradle ecosystem. The library author says he'll start publishing to maven soon... #2020-09-05 17:08:18 seancorfield You can use a local .jar file directly in deps.edn via a :local/root dependency. We use that for a couple of things that have to build locally because they aren't published anywhere. #2020-09-05 17:09:04 Quest oh man, I immediately get what you mean, but how did I utterly fail to find this in a doc somewhere? I spent 20 minutes searching around #2020-09-05 17:09:32 Quest anyway thanks Sean, you're a big help in many channels 🙂 #2020-09-05 17:10:58 seancorfield It doesn't "scale" so well if you're part of a team or want to use it in a CI pipeline, but it's fine for local development and testing. #2020-09-05 17:11:39 Quest Ahh, I'll try just reading through the whole deps guide. I kept searching for "clojure deps include .jar library" and similar terms, never thought of "local library" #2020-09-05 17:13:55 Quest As far as "scaling" this approach, I was just going to git submodule the dependency library.
I could see how many layers of builds would take a long time / cause odd failures. Thankfully still at "fun personal project" size for now #2020-09-05 17:54:25 seancorfield @U22M06EKZ You shouldn't need a pom.xml for this usage -- but if that Java library has any dependencies, you need some way to tell clojure about them (if the JAR contains a pom.xml the t.d.a machinery would take care of that, else you could just add those deps directly to your deps.edn file). #2020-09-05 17:57:33 Quest @seancorfield Hmm, I could uptake them into my deps.edn. Do you know of a way to determine if the jar was loaded or not?
I've just been trying to run an :import on a class that I expect to exist, but wondering it there's a better way to poke around #2020-09-05 18:00:48 seancorfield Use the -Spath option on the clojure CLI to see what the classpath contains. #2020-09-05 18:01:43 seancorfield Or even inside your REPL: (System/getProperty "java.class.path") #2020-09-05 18:02:50 seancorfield user=> (run! println (clojure.string/split (System/getProperty "java.class.path") #":"))
/home/seanc/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar
/home/seanc/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
/home/seanc/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
src
nil
user=>
(example from a bare REPL) #2020-09-05 18:05:25 Quest ^ I figured there was something from the REPL, but -Spath is a good tip. Appreciate the knowledge-share, you had a hand in me starting adoption of deps.edn a year ago too and I doubt I would've tried it otherwise 👍 #2020-09-05 18:07:07 seancorfield Cool. We've never regretted switching to CLI/`deps.edn` in 2018 at work. #2020-09-05 18:08:15 seancorfield (and we're using the bleeding edge prerelease version right now -- as of yesterday) #2020-09-05 17:26:07 practicalli For the up-coming release of Clojure CLI tools:
I have been using clojure -R:cognitect-rebl -A:nrebl to run REBL with nREPL. The -R:congnitect-rebl load in dependencies from an alias that also has a :main-opts (which it not the main namespace to run) and -A:nrebl which has a :main-opts I do want to run.
If I create a separate alias for cognitect-repl without the :main-opts and use clojure -M:rebl-deps:nrebl it works.
Or I could just add all the rebl dependencies into the :nrebl alias.
Does that seem the right approach? #2020-09-05 17:32:18 seancorfield @jr0cket I suspect if you used -A:cognitect-rebl:nrebl it would run the :main-opts from :nrebl since they overwrite each other -- last one wins. #2020-09-05 17:34:32 seancorfield You could also use -A:cognitect-rebl:nrebl -M:nrebl which would work on both today's stable version and the current prerelease version without the -A warning about using -M to run the :main-opts (since the "winning" :main-opts here is using -M). #2020-09-05 17:39:25 seancorfield I think the guidance from Alex has fairly consistently been to separate out :main-opts from other stuff if you need to mix'n'match main opts from several aliases. But the expansion of both -M and -X to respect :extra-deps and other stuff in the latest prerelease feels like the pressure is in the opposite direction -- toward bundling more things into a single alias for convenience @jr0cket Not sure how closely you've been following the discussions generally in this channel? #2020-09-05 18:56:25 practicalli I've read the whole conversation in here and reading through the docs Alex wrote. Still digesting what it all means.
I'm currently testing all the aliases I have to understand the changes #2020-09-05 19:07:37 seancorfield I'm working on updating clj-new to work with -X, much as I've updated depstar. I'll probably add :new-x and :depstar-x aliases to my .clojure/deps.edn file so folks can use those tools via -X as examples. It'll be a while before that can become the default 🙂 #2020-09-05 20:28:52 alexmiller @jr0cket would love to hear your feedback too #2020-09-06 01:12:08 practicalli @alexmiller so far very positive changes. I like that -M feels similar in concept to the :main in Leiningen, in part easing the move from Leiningen to CLI tools.
My current thoughts are around edge cases where I have multiple aliases containing :main-opts and how to use them together with certainty rather than optomism. I have tried to elaborate some options in this gist
https://gist.github.com/jr0cket/6fae7576a87d5d32eda4be2253f41692 #2020-09-06 01:29:56 seancorfield @jr0cket Multiple aliases containing :main-opts is an "issue" today (stable) as well as tomorrow (prerelease), and the answer is the same as it has always been -- per the documentation -- "If multiple maps with these keys are activated, only the last one will be used" -- see https://clojure.org/reference/deps_and_cli#_aliases which explains how each different type of option is "merged" when multiple appear. The same wording is present in the prerelease reference docs, but spread out over multiple sections, rather than gathered together in one section. #2020-09-06 01:30:17 seancorfield Is that not "certainty" enough? #2020-09-06 08:02:14 practicalli Certainly in my mind would be as part of the syntax rather than convention. If the convention changed in a release without requiring a change to the syntax of use.
Convention also means more to remember, especially as some parts of slides are merged and others only take the last. I assume (hope) nothing takes just the first.
Not saying it's an issue, but it adds to the learning curve. #2020-09-06 08:11:21 practicalli As it's a current issue and there isn't anything in the up coming changes to resolve it, then this aspect seems a moot discuss point. I'll make a decision and move on. #2020-09-08 15:52:20 alexmiller fwiw, I don't see this as an issue - the documented behavior is what it does and what I expect it to do in the future #2020-09-05 17:41:13 seancorfield (but of course, at this point, there's still some potential for changes in the prerelease that the -A/`-M` usage recommendations may change) #2020-09-05 17:42:08 hugod I happened to watch Spec’able before reading Alex’s post. Seems pretty ironic. #2020-09-05 17:42:48 seancorfield Spec'able? #2020-09-05 17:43:47 hugod sorry, sepc’ulation #2020-09-05 17:51:53 seancorfield TL;DR: don't break stuff, only add stuff and fix bugs 🙂 #2020-09-05 19:13:17 andy.fingerhut at least once you decide you are out of alpha #2020-09-05 22:45:31 hugod The deps/cli reference page only talks about the library being in alpha. The guide doesn’t mention it at all, and besides “But, that is not to say just leave your thing 0.0.967. At a certain point, you are going to have users, and whether you change it to 1.0 or not, they are going to be depending on your stuff.” #2020-09-05 19:35:26 seancorfield Feedback on the "WARNING: When invoking clojure.main, use -M" message (for @alexmiller) -- this usage produces that warning: clojure -m my.entry.point -- is that intentional? Do you really want folks to write clojure -M -m my.entry.point instead? #2020-09-05 20:19:01 alexmiller Yes #2020-09-05 20:34:34 borkdude Is this to discourage that usage in favor of -X? It seems to be another maybe unnecessary breaking thing that's been there for a long time #2020-09-05 20:35:31 alexmiller It’s not to discourage it as much as that we are trying to open up the repl arg space for other things #2020-09-05 20:35:58 alexmiller And that means making clojure.main usage explicit #2020-09-05 20:41:07 seancorfield Interesting choice. A lot of tutorials out there talk about running simple Clojure projects with just clojure -m my.entry.point #2020-09-05 20:41:47 seancorfield I think beginners will (quite rightly) ask "Why do we have to specify two m options?"... #2020-09-05 20:43:52 alexmiller “Lot” = ? 2 or 3? #2020-09-05 20:44:25 alexmiller And if these tutorials transition into talking about -X... #2020-09-05 20:46:02 alexmiller We want clj to do more. In some cases that means restructuring these options. It’s either that, or clj2 and clojure2 which do not seem better to me #2020-09-05 20:46:02 seancorfield Sure, if all the books and tutorials that exist today are updated to use -X instead of -m, it's not going to be a problem 🙂 #2020-09-05 20:52:40 dpsutton Thanks for balancing all these things in a spreadsheet so lots of concerns are managed. On a Saturday no less. Looking forward to the previews #2020-09-05 21:05:54 seancorfield (I just want to be clear that I think all the new functionality is great -- all of my concerns are around how existing (stable) functionality is deprecated and/or removed over time) #2020-09-05 21:06:50 seancorfield And, ultimately, I'm not going to be the one complaining about the changes since I'm already on the latest prerelease version on every machine and I'm aggressively adopting the new functionality! #2020-09-05 21:08:09 seancorfield (I am somewhat concerned about users of my CLI/`deps.edn`-based projects, of course 🙂 ) #2020-09-05 21:09:21 borkdude There was a time when the clojure CLI wasn't driven by tools.deps.alpha right? Wasn't it non-alpha back then? Isn't only tools.deps alpha but the CLI / shell script now also got alpha? #2020-09-06 00:13:24 alexmiller No, clj and tools.deps were developed together #2020-09-05 21:09:39 seancorfield (`depstar` added support for -X in its most recent 1.1.104 release, last week) #2020-09-05 21:12:20 borkdude I could be misremembering things, but I think there was a clojure package already years ago, before tools.deps.alpha? Not sure what it did back then, but -m was maybe already a part of it back then #2020-09-05 21:14:22 seancorfield I know there was an unofficial clojure script floating around which behaved differently than the official one, once it appeared. #2020-09-05 21:14:59 borkdude maybe it just did java -jar clojure.jar #2020-09-05 21:17:52 seancorfield Yeah, I think that was pretty much it. My recollection of the official CLI was that it appeared in tandem with t.d.a during the 1.9 cycle once Spec was split out of the core clojure.jar since you could no longer just run a REPL with the JAR file alone so it was supporting machinery for that... but I'm having a hard time finding any of the original posts about it (maybe I should scan back through Inside Clojure since I bet Alex blogged about it). #2020-09-05 21:18:21 borkdude ah right, thanks #2020-09-05 21:19:24 seancorfield Hmm, the pre-1.10 Inside Clojure posts are fairly sparse... #2020-09-05 21:22:12 seancorfield (July 2017) #2020-09-05 23:52:45 seancorfield Been playing with the new features, and it's nice that you can have the following
:new {:extra-deps {seancorfield/clj-new {:mvn/version "RELEASE"}}
:exec-fn clj-new/create
:exec-args {:template lib} ; default
:main-opts ["-m" "clj-new.create"]}
and it works with -A:new today (or -A:new -M:new) and, with the prerelease it works with -M:new (or -A:new -M:new without giving a warning) and it also works with -X:new so you can have a single alias provide for both styles of execution. #2020-09-07 15:52:28 practicalli Forgetting migration for a moment, would you see :exec-fn and :exec-args replacing :main-opts in the long term?
Using these new keys seems a nicer approach for the design of the alias code itself.
Would this be an example of edn over strings that Alex mentions? #2020-09-07 16:47:43 seancorfield I think we'll see a good uptake on the :exec-fn approach because you can avoid parsing strings to get your arguments and you can easily provide defaults -- configurable defaults, since each alias have have its own :exec-args map. #2020-09-05 23:57:29 seancorfield That's probably what I'll do in my dot-clojure repo by way of education for folks and as a way to encourage folks to transition (or at least see what the options available are). #2020-09-06 01:18:25 hugod I like the new features. :ns-default and :ns-aliases look very useful. The -P option is great, and could be even nicer if it (optionally) output a map of the resolved dependencies, which could be used to make reproducible builds. #2020-09-06 01:40:59 alexmiller Every build outputs a lib map in the cache dir with this info (-Sdescribe tells you where) #2020-09-06 01:49:08 hugod didn’t realise that 🙂 #2020-09-06 01:18:43 hugod To be honest, I’ve always found the different merging rules confusing and never really understood their purpose. Why not just have -A to specify the aliases to merge, and -e, -x (equivalent of the :exec-fn key) or -m (equivalent of the :main-opts key) to specify or override what should be run. Maybe with a -r for a future repl specifier.
If the config is not specific, define a precedence for :exec-fn and :main-opts.
This would separate (decomplect, dare I say?) merging of aliases from specifying what to invoke, which I, for one, find simpler to understand. #2020-09-06 01:41:03 seancorfield @hugod In the stable version, -M didn't pick up any items from the alias(es) except :main-opts and in the first version of -X on the prerelease track didn't pick up any items from the alias(es) except the fn/args spec as I understand it. In the latest prerelease, both of those pick up everything in the alias(es) specified and then -M runs the :main-opts while -X runs the function invocation -- and -A is slated for being similar (picking up everything in the alias(es)), but ultimately just running the REPL, not executing main/function stuff. That's why I proposed keeping But, either way, it's a much more streamlined piece of logic: all three options pick up all options and then each one of -A (or maybe -R?), -M, and -X selects what execution model is used. #2020-09-06 01:41:33 seancorfield It's nice and streamlined. And I think it removes a lot of confusion about which parts of an alias get picked up by which options. #2020-09-06 01:50:29 hugod Having -M, -X, and -A pick up everything certainly seems like an improvement. I would still find it simpler to just have the one option for specifying aliases, and then options for specifying what to run. Using lower case for the latter would allow for the cli to maintain backward compatibility iiuc. #2020-09-06 01:52:13 seancorfield I suspect there would be confusion, since -e, -m, and -r are existing main options today (for clojure.main). #2020-09-06 01:55:19 seancorfield You'd also need to specify both an option to collect aliases and then an additional option to specify the action to take. With the prerelease direction, you'd only need one option: either -X for aliases and execute function, or -M for aliases and main function, or -A for aliases and the REPL (at some point, given -A currently still executes main opts). #2020-09-06 02:03:31 hugod I was suggesting -e and -m kept their current meaning, and implied invoking clojure main. #2020-09-06 02:04:48 seancorfield But :main-opts can contain a combination of those so I don't think that's going to be intuitive. #2020-09-06 02:05:05 seancorfield (`:main-opts` can also contain -i and -r options) #2020-09-06 04:59:43 mike_ananev @alexmiller Is it possible to add functionality of generation distributionManagement section to 'clj -X:deps mvn-pom' command?
I need to distribute my artefacts written in Clojure to corporate Nexus. To do this I need section in pom.xml:
<distributionManagement>
<repository>
<id>releases</id>
<url></url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<url></url>
</snapshotRepository>
</distributionManagement>
Building and deploying artefacts is performed in corporate Jenkins, where I can't manually add this section. #2020-09-06 05:02:03 seancorfield @mike1452 Since clj -X:deps mvn-pom only updates dependencies and a few other bits, so what I tend to do is generate an initial POM, augment it manually, check it in to version control, then just run that command as needed to update the dependencies section (and it leaves all the rest intact). #2020-09-06 05:02:28 seancorfield So you could manually add that section, and clj will not overwrite it. #2020-09-06 05:34:48 vlaaad Weekly reminder that reading/updating pom.xml is a workaround for the problem, not a solution :) #2020-09-06 07:24:12 borkdude @alexmiller I know tools.deps isn't going to expand ~, but I'd like to know the reason for this. Reason I'm asking is that I'm considering doing this in .clj-kondo/config.edn and would like to know if I'm missing some problems #2020-09-06 11:46:24 alexmiller Because then everyone reading deps,edn has to interpret the file #2020-09-06 11:47:31 alexmiller It’s no longer just edn/read, it’s that plus run magic code #2020-09-06 12:32:40 borkdude Why does it have to be interpreted by anything else than tools.deps which already does interpretation of this entire model? #2020-09-06 12:37:17 borkdude I mean, merge, assoc, etc still works for other tools in this case? #2020-09-06 12:45:56 alexmiller If you have alias data, anyone may be reading it via the basis (without tools.deps, which has a lot of deps) #2020-09-06 12:47:41 alexmiller There are a bunch of tools already reading and manipulating deps.edn. The less they have to know, the better #2020-09-06 13:30:12 borkdude I don’t think this will be a problem for the linter config file. Thank you sir. #2020-09-06 09:31:19 mike_ananev @seancorfield Is it possible to exclude clj source files from uberjar using depstar? I need only aot compiled files in uberjar, when distributing it. Also, how do you recommend to compile java source files, when using depstar? #2020-09-06 09:52:37 borkdude Ah no, scratch that, that's only for removing something from the classpath. #2020-09-06 09:54:02 borkdude Ah, and that's exposed via -X (not clojure's -X, but depstar's -X) #2020-09-06 09:54:27 borkdude so probably something like -X "\.clj$" would work #2020-09-06 12:53:27 dominicm You don't need to remove the clj files anyway. Or is there an additional piece of information? #2020-09-06 18:33:06 seancorfield The regex-based file exclusion was added in the most recent depstar release @mike1452 -- did the suggestions above work for you? #2020-09-06 18:34:03 seancorfield (you could also specify a classpath that only had the class path, without the src path I suppose, if every piece of source has been compiled?) #2020-09-06 18:35:00 seancorfield As for Java, that's outside the scope of a Clojure tool: compile it yourself and then add that to the classpath (depends on how/where you compile your Java source to .class files). #2020-09-06 22:02:13 dominicm How can I remove org.clojure/clojure from the classpath? I seem to recall a trick to it. #2020-09-06 22:03:34 dominicm (I want to use org.clojure/clojure$sources instead) #2020-09-06 22:39:52 alexmiller Just specify a different coord for it? Or use :override-deps? #2020-09-06 22:41:12 alexmiller Keep in mind that everything calls clojure.main and you need a class for them jvm to invoke #2020-09-06 22:45:47 dominicm I tried using override-deps, maybe I used it wrong. #2020-09-06 22:46:52 dominicm Error building classpath. Coordinate type not loaded for library org.clojure/clojure in coordinate {} #2020-09-06 22:57:35 dominicm (I also tried nil) #2020-09-06 23:25:36 seancorfield You need an alias to use :override-deps #2020-09-06 23:26:54 dominicm I used one. But I'm trying to override to remove it and {} didn't work at that removal coordinate. #2020-09-07 00:46:47 seancorfield Can't you override it to point to local JAR? The sources JAR file? #2020-09-07 01:25:38 seancorfield Since we're getting a lot of depstar questions here, I created a dedicated channel for support/questions/etc. Also for clj-new. #2020-09-07 06:59:22 borkdude @dominicm you can also use classpath overrides with nil. I’ve used this for creating babashka uberjars (see its README and look for uberjar) #2020-09-07 09:53:33 kirill.salykin goodmorning,
please advice how I can turn off the OmitStackTraceInFastThrow?
clojure -A:app -J "XX:-OmitStackTraceInFastThrow"
doesnt seem to work #2020-09-07 09:54:32 dharrigan clojure -J-XX:-OmitStackTraceInFastThrow #2020-09-07 09:54:46 kirill.salykin thanks! it is all about the order #2020-09-07 09:55:04 dharrigan np #2020-09-07 10:28:42 practicalli In clojure pre-release (1.10.1.672) it seems that the -X flag does not download dependencies specified in the alias :extra-deps section. Is this the correct behavior?
From my interpretation of "-X now supports multiple aliases, aliases with other argmap options like :extra-deps, and ad hoc functions" I would have thought deps would be downloaded #2020-09-07 10:29:30 borkdude is that in combination with -A? #2020-09-07 10:29:59 practicalli No, just by itself. I understood -A was being depreciated, so am trying to understand the scope of using -M (which seems fairly clear) and -X which I am still confused about (from a users ponit of view) #2020-09-07 13:30:58 alexmiller @jr0cket that is intended to work, so could be a bug #2020-09-07 15:24:27 practicalli Yes, it does work on a clean install of Clojure CLI with the ~/.clojure/.cpcache removed. This is making much more sence now. Thanks. #2020-09-07 13:45:26 rickmoynihan To temporarily swap out a git dep for a local dev version, am I normally better adding an alias with :override-deps in it to a :local/root, or adding :classpath-overrides to my ~/.clojure/deps.edn ? #2020-09-07 13:48:24 dominicm @rickmoynihan I usually use an override-deps (actually extra-deps because it's the same otherwise) in ~/.clojure/deps.edn because:
• Nobody else can use it - has my homedir in
• Most people on my teams aren't contributing to those other libraries anyway. It's less reusable than one would think
#2020-09-07 13:50:25 rickmoynihan yeah for both options I’m talking about adding it to ~/.clojure/deps.edn… trying to understand which way is better. #2020-09-07 13:51:11 rickmoynihan One (`:override-deps`) effects resolve-deps and the other make-classpath right? #2020-09-07 13:52:09 rickmoynihan Practically though; aren’t they both equivalent? #2020-09-07 13:52:16 dominicm I think for your case, they're essentially equivalent. I'd go with extra-deps just because it's more famiilar. #2020-09-07 13:53:34 rickmoynihan except :classpath-overrides assumes the lib only introduces a single classpath root, is that right? #2020-09-07 13:54:06 rickmoynihan so e.g. it would potentially miss including resources etc #2020-09-07 13:55:13 dominicm Exactly, yeah #2020-09-07 14:55:50 borkdude Re command line args, I have one tool that only has one CLI argument, --opts in which you pass a Clojure map. For that tool I think it made sense and it also simplified command line parsing, possibly also easier to keep it non-breaking. I'm not suggesting that other tools do this, but works for this one particular thing.
clojure -A:carve --opts '{:paths ["src" "test"]}' #2020-09-07 23:05:42 sogaiu i have a tool that does something similar -- though it doesn't use anything like --opts before the string.
it's nice to not have to decide on command line arguments when you don't know what option and arg names are likely to be used more often. #2020-09-07 15:03:25 dominicm I suppose the clj way of doing that is would be clj -X:carve :paths '["src" "test"]'? #2020-09-07 15:04:22 borkdude I guess so, but if we're going to use quotes, why not quote the whole thing anyway #2020-09-07 15:05:14 dominicm I mention it only because I think this is what clj is trying to make the standard for cli tools. While I personally think --path src --path test would be the best, standards are better than better. #2020-09-07 15:08:32 borkdude or :paths src,test since comma is whitespace in Clojure, but not in bash? #2020-09-07 15:08:54 borkdude $ bb -e "*command-line-args*" -- :paths src,test
(":paths" "src,test")
#2020-09-07 15:09:40 dominicm You're thinking in Clojure 🙂 I'm thinking of bash. Admittedly, my answer was without context. Generally I'd much prefer an api where paths was the & args at the end so I could just do:
clj -X:carve dev-*
#2020-09-07 15:09:59 dominicm (which expands to clj -X:carve dev-src dev-resources dev-etc) #2020-09-07 15:12:18 borkdude If bash would expand that comma separated, then it would work as passing one value. #2020-09-07 15:12:42 borkdude Anyway, it doesn't. #2020-09-07 15:14:06 borkdude Yeah, if keywords would separate other values then :jars *.jar :paths src* would work #2020-09-07 15:14:30 borkdude unless your dir starts with a colon :) #2020-09-07 15:15:55 borkdude but I guess using a comma as explicit separator works:
$ bb -e "*command-line-args*" -- :foo t* , :nums 1 2 3
(":foo" "tags" "test" "test-resources" "," ":nums" "1" "2" "3")
#2020-09-07 15:21:41 borkdude This doesn't work in powershell (no expansion, comma is gone)
PS Microsoft.PowerShell.Core\FileSystem::\\wsl$\Ubuntu-20.04\tmp> cd C:/Temp
PS C:\Temp> bb -e "*command-line-args*" -- :foo t* , :nums 1 2 3
(":foo" "t*" ":nums" "1" "2" "3") #2020-09-07 15:22:25 borkdude Finally, cmd.exe (no expansion, preserves comma)
C:\Temp>bb -e "*command-line-args*" -- :foo t* , :nums 1 2 3
(":foo" "t*" "," ":nums" "1" "2" "3") #2020-09-07 15:36:41 dominicm No idea how cmd.exe handles wildcards. Ultimately there's some platform specific tweaks to match the ergonomics of a platform you're on. #2020-09-07 15:38:41 borkdude I guess there's a reason WSL2 is getting popular #2020-09-07 15:40:49 dominicm But also - this is what people on cmd.exe expect. That's fine and while I don't understand it, if that's what makes sense for their OS then fine. #2020-09-07 15:42:22 borkdude I listened to a podcast by someone who is working on the new Terminal app. She said that they couldn't change cmd.exe's behavior because that would be breaking for lots of people. But everyone basically agrees that it sucks. #2020-09-07 15:43:41 borkdude Making the cmd.exe window smaller will make it faster, because there is less rendering, which is blocking #2020-09-07 15:44:04 borkdude ok sorry, got a little bit off topic #2020-09-07 17:49:16 lread Getting back to clojure/clj command line arg compatibility, this is not helpful right now, but I am curious. If we could go back in time would these tools have been better initially named something like clj-alpha1 and clojure-alpha1? This would have drawn attention to their alpha-ness and also allowed a path for introducing breaking changes. #2020-09-07 17:51:28 lread Then again maybe not… this would have implied clojure itself is alpha. #2020-09-07 17:51:43 lread Tricky little problem. #2020-09-07 17:55:02 borkdude The help could have mentioned the word alpha at least somewhere. From and end user's perspective, it's hard to see this is alpha, you'd have to know what .jar it's calling underneath which has alpha in the name. This entire page doesn't mention alpha neither: https://clojure.org/guides/deps_and_cli #2020-09-07 17:59:08 seancorfield I just got a bug report against the new version of clj-new and it's obviously broken but it leads me to a question about t.d.a and migrations: clj-new uses t.d.a to resolve coordinates for templates and it was using t.d.a 0.8.677 and the clojure.tools.deps.alpha.reader namespace with default-deps and read-deps. I updated it to 0.9.782 and switched the code over to pull in the runtime basis... but of course that doesn't exist in the new CLI! Doh! I could revert to 0.8.677 and continue using the old logic, but I was wondering if there was a way in 0.9.782 to get at that same logic so that utilities could run with the basis, if it exists, else fall back to the equivalent of (read-deps (default-deps))? Mostly a question for @alexmiller I guess... #2020-09-07 18:15:58 alexmiller Those functions now exist in the tools.deps.alpha namespace #2020-09-07 18:16:18 alexmiller Or their equivalents - there were some name changes etc #2020-09-07 18:18:22 seancorfield Hmm, it looks like some of the logic for finding deps.edn has moved into the CLI script, and is no longer in t.d.a? #2020-09-07 18:18:49 alexmiller No, should all be there #2020-09-07 18:19:00 alexmiller Shape is a little different #2020-09-07 18:19:06 seancorfield OK, I'll keep digging... #2020-09-07 18:19:50 alexmiller Sorry, not at a computer to be precise #2020-09-07 18:21:00 alexmiller It’s just root-deps and user-deps and ./deps.edn #2020-09-07 18:21:23 alexmiller Read and merge #2020-09-07 18:32:54 seancorfield find-edn-maps was what I needed so I think I'm good now... #2020-09-07 18:52:23 seancorfield For anyone else working on t.d.a-based tooling who wants to leverage 0.9.782 but continue to support older CLI versions, here's what I ended up with for the basis:
(delay (if-let [basis-file (System/getProperty "clojure.basis")]
(-> basis-file
(io/file)
(slurp)
(edn/read-string))
(let [{:keys [root-edn user-edn project-edn]} (deps/find-edn-maps)]
(deps/merge-edns (filterv some? [root-edn user-edn project-edn]))))))
That's to replace the equivalent of (read-deps (default-deps)) from t.d.a 0.8.677 -- it's not identical, but for the purposes of equivalence with 0.8.677 it seems close enough. #2020-09-08 02:11:42 cfleming Is there any way with deps to manage source artifacts for non-Clojure dependencies? #2020-09-08 02:12:25 cfleming Or rather, how could I specify that I would like them to be downloaded, and how would they be distinguishable from binary jars in the output? #2020-09-08 02:25:00 cfleming This seems to be the answer:
{:deps {junit/junit {:mvn/version "4.13"}
junit/junit$sources {:mvn/version "4.13"}}} #2020-09-08 02:30:05 cfleming They do end up on the classpath, which is unfortunate. I guess they could go in an alias. #2020-09-08 03:15:05 cfleming When using :local/root, the doc says:
> If the jar has been packaged with a pom.xml file, the pom will be read and used to find transitive deps
Is there a way to prevent this from happening? I would just like it to use the jar I give it and not look for transitive deps. #2020-09-08 03:22:01 cfleming Looking at the code, it seems like not. My use case here is trying to use deps to manage the dependencies for Cursive. I need to depend on the IntelliJ framework, so I have a script which downloads and unpacks that, and I’m trying to create a deps.edn which refers to the jars within it. However a bunch of those are third party jars which include poms. #2020-09-08 03:22:26 alexmiller you could use :local/root to a directory which has a deps.edn that includes the jar on the :paths #2020-09-08 03:23:06 cfleming Oh, I can just put jar paths directly in :paths? That’s even better. #2020-09-08 03:23:16 alexmiller kind of a hack, but it works #2020-09-08 03:23:24 cfleming I guess that makes total sense, but never occurred to me. #2020-09-08 03:23:30 alexmiller :paths are just things that get added to the classpath #2020-09-08 03:23:51 alexmiller on the source stuff, tools.deps is about making classpaths so that's what you're going to get #2020-09-08 03:24:02 cfleming Right. It’s actually much better for this use case since I don’t need to create a bunch of bogus dep symbols. #2020-09-08 03:24:38 cfleming Ok. Using $sources works fine, so I’ll support that in Cursive, document how to do it and suggest putting them in an alias. #2020-09-08 10:24:36 practicalli @alexmiller I'm confused about using -M with a project rather than a specific alias.
In this example I use the hello-world example from https://clojure.org/guides/deps_and_cli#_writing_a_program
A project deps.edn file was created containing the dependency for clojure.java-time and the source code from that page copied into src/hello.clj
clojure -m hello runs the project and returns the time from running the -main function.
However this gives a warning:
WARNING: When invoking clojure.main, use -M
clojure -M runs a REPL
clojure -M -m hello runs the project and returns the time. But then I ask myself what is the purpose of -M
Creating an alias to run the project seems an interesting idea, as I could also set default arguments. I can only get this to work with :main-opts and not :exec-fn
This version of the alias works when calling clojure -M
:aliases
{:run-project {:main-opts ["-m" "hello"]}}
This version of the alias simply runs the REPL
:aliases
{:run-project {:exec-fn hello]}}
Am I missing something from :exec-fn ?
Edit: yes I am.
The -X option is for running :exec-fn and not the -M function, oops.
So clojure -X:run-project should work... except the hello-world project has an unqualified function and cannot be resolved.
Moving the source code to src/practicalli/hello.clj and calling clojure -X:run-project gives an execution error, (ArityException) as the -main function does not take any arguments (defn -main [] ,,,) . Changing the -main function to (defn -main [& args] ,,,) fixes that and calling clojure -X:run-project works.
From a user point of view, that seems quite a lot to extrapolate from the original warning message when running a project with clojure -m hello #2020-09-08 14:49:26 alexmiller The guide hasn’t been updated for the new stuff yet - I’ll do that with the stable release of it #2020-09-08 14:57:41 practicalli Great. That time frame wont impact me at all. Thanks If you want help with the docs, let me know (although you are the expert here) #2020-09-08 11:57:50 practicalli Thinking about this further, it does feel that the -X flag would be the most important flag in the long term, especially with :exec-arg allowing default arguments that can be over-written on the command line.
-M seems to be more of a (very useful) convenience function for the specific case of running -main-opts configurations. This does remove the need for command line args and bash quoting, so that is valuable. -M is also useful for those aliases that dont work with -X , eg rebel-readline I could only get working with -M .
As more maintainers adopt the new version of Clojure CLI, there is scope to configure all projects and tools to work with -X and eventually make -M redundant.
Using the -M flag seems to help a little when making the change from Leiningen to CLI tools, as setting the main namespace is a familiar concept.
I am sure I am over-simplifying this and probably missing out on a lot of understanding as of yet. #2020-09-08 12:42:42 hugod Would -M not continue to be useful for utilities that you expect to be invokable from the shell? How would you allow passing glob’ed files for example? #2020-09-08 12:48:57 practicalli Yes, it seems the -M option is valuable when the :main-opts data is more intricate. Interesting to see what can be done with :exec-args verses :main-opts .
I do like :exec-args as a means of using edn over strings.
I dont believe I've have any use cases for using glob'ed files, so cant comment on that without specific examples. I assume something like *.jpg would just be a command line argument after the -X:alias-name and passed to the function. #2020-09-08 13:04:53 hugod iuuc, -X expects arguments to be key-path edn-value pairs, so clojure -X:my-alias :x "1 2 3" ends up passing {:x 1} to the exec-fn. #2020-09-08 13:18:13 hugod In general, if you want something that can be be composed with output from tools that don’t produce edn, I think -M is the easier (only?) option. #2020-09-08 13:18:31 practicalli Ah, so I assume clojure -X:alias :x '["1 2 3"]' is required for :exec-fn , which fits in the concept stucture over strings. Although more specific options seem preferable.
clojure -X:alias :x 1 :y 2 z: 3 .
This is less ambiguous and ties into the general approach I use for Clojure arguments. #2020-09-08 13:18:57 practicalli Still lots to learn, so interesting to hear these thoughts. Thanks. #2020-09-08 13:19:43 hugod I’m still trying to grasp the implication of the changes too. #2020-09-08 14:18:52 alexmiller @U05254DQM I think mostly what you’ve gleaned is correct. -M is for pass through to clojure.main. (And originally that’s the only thing clj did.) Calling with no exec opt is for repl. Right now that’s done with clojure.main but you should consider that an implementation detail. -X is for the new function execution style invocation. #2020-09-08 14:20:46 alexmiller We expect most tools are better off with -X, but totally possible to support both of course #2020-09-08 14:23:18 alexmiller For your own app, not sure yet how this will be taken up. Using -X means you are at least somewhat bound to using clj as your launcher (as the exec stub code is not available anywhere else (yet)). Rich seems to think that’s fine, I’m a little less sure about it. :) #2020-09-08 14:25:20 alexmiller My biggest issue with the -X arg passing is that passing a string path as the v requires wrapping in multiple quotes which kind of ruins - expansion and other glob and auto completion stuff. I’d say that’s not totally finalized #2020-09-08 14:43:58 practicalli Thanks for confirming Alex. I am far more confident in my understanding of the changes. Apart from updating practicalli/clojure-deps-edn with either -M or -X variants instead of -A, there seems no impact on my development projects.
My notes from a project developer are shared on this gist: https://gist.github.com/jr0cket/ae7dd745eb45870109ace59fe835ce80 #2020-09-08 14:47:22 practicalli Updating the Practicalli books with the new approach is pretty easy and can run with the 'new' and 'classic' options for a while.
I've planned screencasts of using Clojure CLI tools for Clojurists Together over the next 2 months. I would rather not re-do them as high quality screencasts are time intensive. I will need to decide when to do these videos, so any thoughts on a release date for the new features would help my own planning (I appreciate any comments on dates are no guarantee).
If a release is not planned until December, I'll opt for some lower quality screencasts and recreate them running up to a release. #2020-09-08 14:48:31 alexmiller I’m hoping to get to a new stable release in the next week or two #2020-09-08 12:43:33 jeremys Hi, I have a couple questions about deps API. There is there a complement function to slurp-deps that also merges the 'user wide' deps.edn and the 'system wide' one ? Also the docs says that resolve-deps and make-classpath are supported. Then the question is: is make-classpath-map to be considered private api ? (and so subject to change?) #2020-09-08 14:11:29 alexmiller You can use find-edn-maps to read and merge-edns function to merge edn maps once they’ve been read #2020-09-08 14:12:29 alexmiller make-classpath-map is public but some details of the make-classpath details are going to change, probably this week #2020-09-08 14:13:06 alexmiller (Regarding path ordering) #2020-09-08 14:16:02 alexmiller The whole make-classpath area is in a bit of transition #2020-09-08 14:25:26 jeremys Ok Alex, thanks a lot! I have been foolishly experimenting with the idea of making https://github.com/JeremS/mbt based on tools deps and I ran into those questions today. If I may I have one more concerning tools.gitlibs. Right now it uses jgit v4. Do you plan to use v5? #2020-09-08 14:31:12 alexmiller Yeah, I’ve spent some time on it but not enough to get it over the line. It also plays into the jsch ssh stuff and look at their newer impl with Apache Mina. #2020-09-08 14:31:46 alexmiller Just FYI, I have built a tools.build tool that will hopefully be making its way toward release soon #2020-09-08 14:43:11 jeremys Oh man... my rationale was that tools.deps begs a tools.build... A proper tools.build is very good!
The jgit question was because the clj-git project uses the v5 making it, I believe, incompatible with tools deps
(breakage between jgit 4 and 5)
Thank you again for the info. #2020-09-08 17:25:48 seancorfield Re: find/read/merge #2020-09-08 19:38:06 jeremys Thank you @seancorfield the last 2 lines are almost exactly what I need! #2020-09-08 15:04:34 borkdude Maybe aliases should also become fully qualified? #2020-09-08 15:04:56 borkdude Then we can have a global (worldwide) alias registry =) #2020-09-08 15:05:45 borkdude I wasn't trolling (at least, not consciously) #2020-09-08 15:05:58 mpenet I didn't mean it that way 🙂 #2020-09-08 15:07:44 borkdude e.g. want to use seancorfield's aliases -A:org.seancorfield/rebl #2020-09-08 15:14:43 borkdude I guess the problem with that is that aliases can change over time and effectively become a dep. Importing aliases from libs might be an idea, but maybe a bit too much. #2020-09-08 16:02:26 alexmiller aliases can be fully qualified now #2020-09-08 16:02:58 alexmiller people typically don't because more typing, but that's fully supported #2020-09-09 08:39:51 rickmoynihan I’ve been doing this for a while with local overrides, e.g. :local/libname will be defined in ~/.clojure/deps.edn for when I have a local checkout of it. I think it works quite well, also to distinguish user wide aliases from project ones.
cc @borkdude #2020-09-09 08:41:29 borkdude yes, doing something something similar #2020-09-08 16:04:04 alexmiller one thing we've done a bit of thinking about is - how could I specify a tool to use and have it automatically pulled from github without needing to declare an alias at all #2020-09-08 16:04:54 borkdude yes, that was a bit was I was thinking with this #2020-09-08 16:05:08 borkdude bit like lein new does maybe #2020-09-08 16:05:52 borkdude Aliases can be fully qualified now, but will it ever be deprecated that they aren't - as with deps #2020-09-08 16:05:55 dominicm Need some aliases procurers #2020-09-08 16:06:03 alexmiller probably not going to fully tackle that for a bit, but I think it would be pretty useful. you need some kind of naming convention that takes you to a gh repo, that can be a deps.edn with deps etc #2020-09-08 16:06:20 alexmiller I think just github support would take you pretty far, but yes all that needs to be thought about #2020-09-08 16:06:37 alexmiller there is also some open space I've been working in around discovery with -X #2020-09-08 16:06:38 borkdude this can probably be solved in a tool which you can put in your aliases #2020-09-08 16:07:17 alexmiller like could -X have support for listing the functions available to call #2020-09-08 16:07:27 borkdude I wasn't trolling! geez, what is it with people today ;) #2020-09-08 16:07:42 alexmiller how can you support automatic documentation of a -X compatible function #2020-09-08 16:08:00 borkdude maybe metadata? #2020-09-08 16:08:07 borkdude ^:x-callable #2020-09-08 16:08:56 alexmiller all tbd, but with combination of mapping from alias to tool repo, and support for :ns-default (to find the default tool namespace), and ability to discover the -X functions, and help for each option.... #2020-09-08 16:09:11 alexmiller that's an ecosystem #2020-09-08 16:09:31 borkdude yes, the mapping from alias to tool repo was what I was getting at earlier #2020-09-08 16:09:36 alexmiller and then if only had a build system that could make use of those functions... #2020-09-08 16:10:25 borkdude build system? #2020-09-08 16:10:35 borkdude troll #2020-09-08 16:10:41 alexmiller :) #2020-09-08 16:11:04 dominicm Metadata wouldn't work by itself, you'd have to require every namespace. #2020-09-08 16:11:05 alexmiller anyhow, I'm on "vacation", so back to vacating... #2020-09-08 16:11:13 borkdude enjoy! #2020-09-08 16:11:18 dominicm Yes. Begone! #2020-09-08 16:14:18 borkdude Maybe aliases aren't the right fit for name -> github repo and some other concept/option should be invented for that. #2020-09-08 17:27:56 seancorfield Sort of related to this line of thinking: clj-new supports templates on GitHub via naming conventions, so maybe have a look at the readme for the usage and see if it is close to what you are thinking @borkdude? #2020-09-08 17:31:19 borkdude @seancorfield You mean:
clj -A:new -M:new ...
? #2020-09-08 17:33:49 borkdude or do you have some short form where you only mention the org + repo name, with automatically resolving the newest sha? #2020-09-08 17:43:25 seancorfield Currently, you need the full URL but that's really only to detect what type of template you're using. If you know it's GH, you could take just the org/repo-name and build the rest. And my dot-clojure file has an example of using tools.gitlib to figure out the HEAD of master SHA. #2020-09-09 00:24:21 cfleming Just double checking - there is no way to get the deps of a subproject specified via :local/root including deps from an alias, correct? i.e. I cannot specify an alias together with :local/root in some way? #2020-09-09 00:26:37 seancorfield Nope. #2020-09-09 00:26:56 seancorfield Dependencies are opaque in that respect. #2020-09-09 00:28:06 seancorfield Although... I guess you could read the basis (in the prerelease CLI) and figure out the actual :local/root path and see if it has deps.edn and then use t.d.a to read that and then inspect its aliases... 👀 #2020-09-09 00:29:12 seancorfield But a :local/root could point to a JAR, or a directory with pom.xml. Or project.clj 🙂 #2020-09-09 00:32:54 cfleming Right. #2020-09-09 00:33:27 cfleming This is just specifying deps in a deps.edn file, so I can’t do anything funky or programmatic. #2020-09-09 01:13:26 hugod What is the rationale for deprecating unqualified dependency symbols in :deps? (couldn’t see one in the release notes) #2020-09-09 01:18:46 seancorfield (IOW it was deprecated in an earlier prerelease) #2020-09-09 01:20:25 hugod Thanks - I missed that release 🙂 #2020-09-09 01:25:21 seancorfield I spent the first week of August, scurrying around, fixing all our unqualified lib names at work, and all my open source projects! 🙂 #2020-09-09 02:31:32 cfleming Is there any way to substitute values into a deps file that I’m not aware of, e.g. from env vars or properties? In my case I would like to be able to substitute a global version in several deps files. I’m aware of default-deps, but I’d need to be able to substitute it in a :local/root path. #2020-09-09 02:33:31 seancorfield Nope. No substitutions. #2020-09-09 02:39:06 seancorfield (I'm just a bundle of "nope" today, aren't I?) #2020-09-09 02:40:34 cfleming Haha #2020-09-09 02:40:51 cfleming No worries, that was what I expected, just makes my life a little more tedious 🙂 #2020-09-09 13:19:41 hugod Fwiw, having played with the pre-release, I still think the following CLI switches would be simpler, as they separate the selection of aliases from what to execute, and would simplify the doc. A given set of aliases can be used for main, exec or repl, so it makes it easier to compose a command line from other tools if the passing of options is distinct from the selection of what to invoke.
clj [clj-opt*] [-A:aliases] [init-opt*]
clojure [clj-opt*] [-A:aliases] -x [a/fn] [kpath v]*
clojure [clj-opt*] [-A:aliases] [init-opt*] [main-opt] [arg*]
clojure [clj-opt*] -P [other exec opts]
or it could be
clojure [clj-opt*] [-A:aliases] -M [init-opt*] [main-opt] [arg*]
If that is really required, though I confess I don’t see the rationale for having to specify the -M given that you’re passing -m or have a :main-opts key.
The (at least my perceived) lack of usage of the current -T, `-R`, `-C`, and -O options supports the above. #2020-09-09 13:21:43 alexmiller Sorry, not doing that #2020-09-09 13:29:13 hugod That’s clear 🙂 #2020-09-09 16:15:15 seancorfield Also, I think there's more usage of -R out there than you suspect @hugod #2020-09-09 16:22:04 hugod I’m sure I don’t have a good perspective on that, but don’t think it really changes my view that just using -A would be conceptually simpler. #2020-09-09 16:22:30 hugod Given the cli changes seem fixed, I’ll just go and update my nascent build tool invoker project to cope with multiple versions of the cli. #2020-09-09 16:28:13 seancorfield I still think that a) given the expansion of -M to include more than just :main-opts b) the overlap between -A and -M (both run :main-opts now) c) the current uses of -R (doesn't run :main-opts but does "most" of what -A does -- based on resolve args being the most common; it would make more sense for -R to be kept and expanded as the "REPL" usage (and quietly undocumenting -A). So we'd have -R (REPL), -M (main), -X (exec) and they all three respect all arg types in deps.edn and there would be no overlap on :main-opts or :exec-fn. #2020-09-09 17:20:45 hugod I do find it unfortunate that the deps.edn isn’t authoritative. By this I mean that whatever you specify in deps.edn you still have to know whether to invoke it with -M or -X. #2020-09-09 17:30:08 seancorfield That works with -A today and either -M or -X tomorrow! #2020-09-09 17:36:58 hugod I don’t think that is quite what I meant. As a deps.edn writer I would like to provide an abstract, say test, alias for doing something, and have the user be able to invoke that independent of how I chose to implement that alias. Writing it both ways seems like duplication. #2020-09-09 18:55:17 alexmiller well the arg structures are different, not sure how that could work #2020-09-09 19:45:29 seancorfield Exactly, as a tool writer you need to specifically support -M style invocation and/or -X style invocation as two separate functions in your code: -main taking & args which are strings and some-exec-fn taking a single hash map as its argument. #2020-09-10 13:04:41 hugod main obviously takes strings and an exec-fn wants an edn value, but that doesn’t need to dictate the format of arguments passed on the cli. The cli could print edn to strings before passing it as a main argument, or read literal strings to edn strings before passing to an exec-fn. As a strawman, with intentionally hideous names, consider this:
i) default to passing args as literal string values, or add a --string switch to explicitly specify this
ii) add an --edn switch, that causes the next argument to be read as EDN and passed as a value to an exec-fn or as a string to a main.
iii) add a --patch switch that takes an edn path vector and an edn value and patches the default value. if the value is prefixed with --string treat it as a string value rather than EDN. Allow main to have default arguments specified, in the same way as exec-fn.
iv) add a --rest switch that collects the remaining arguments into an EDN vector. --rest-strings could build an EDN vector from string arguments rather than edn values. This could be used to pass glob values, for example.
v) allow an exec-fn to take multiple arguments.
As described this is unwieldy, but if there were volition, I’m sure it could be improved upon. The main idea would be to divorce the specification of the format of the arguments passed on the command line from how they are passed to the implementing function. #2020-09-09 19:08:13 jeremys Hi, I have been working on a https://github.com/JeremS/mbt based on tools.deps for some time. It has a mix of functionality inspired from several other projects and it's at a stage where I use it to version, build and deploy my projects. Maybe someone other than me might find it useful! Cheers #2020-09-09 19:37:57 seancorfield @alexmiller It looks like clojure -X:deps mvn-pom now adds the following dependencies to all pom.xml files:
<dependency>
<groupId>org.clojure</groupId>
<artifactId>tools.deps.alpha</artifactId>
<version>0.9.782</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-nop</artifactId>
<version>1.7.25</version>
</dependency> #2020-09-09 19:39:04 seancorfield That's going to cause a lot of projects to accidentally end up depending on tools.deps.alpha (and the slf4j-nop dependency is likely to be unwelcome in many cases I suspect). #2020-09-09 19:40:45 seancorfield These deps are in a freshly-generated pom.xml file -- and they are also added to any existing pom.xml, potentially overwriting the dependency for any projects that use a different version of t.d.a #2020-09-09 19:43:19 alexmiller ah, interesting #2020-09-09 19:43:51 alexmiller thx, will fix #2020-09-09 19:59:43 seancorfield How will you fix it? For example, my clj-new project does depend on t.d.a (and has a dependency on slf4j-nop because of that) -- but I know that's an outlier -- so you'd need to also not remove any existing dependency on those? #2020-09-09 20:00:51 alexmiller I'll fix it by calculating the basis, not using the injected basis #2020-09-09 20:01:02 alexmiller basically what it did before #2020-09-09 20:02:56 seancorfield Won't that break anyone currently using clojure -A:some:aliases -Spom ? #2020-09-09 20:11:12 alexmiller it doesn't have to - the injected basis includes those argmaps #2020-09-09 20:11:36 alexmiller the calculated basis can use the project deps.edn + the injected basis argmaps #2020-09-09 20:25:43 seancorfield Re: basis -- so -A and -X are distinguishable via the basis? Or am I misunderstanding? #2020-09-09 20:26:57 alexmiller oh, I see what you're asking. well let me think about it more #2020-09-09 20:12:30 alexmiller another option would be to take those additional aliases as args to the -X pom function #2020-09-09 20:12:41 alexmiller that would be an all new thing though #2020-09-09 20:13:24 alexmiller very similar thing exists in the datomic ion-dev tool which I was just helping with a bit :) #2020-09-09 20:14:04 alexmiller I am planning to reinstate the -Spom as well so looking at that right now too #2020-09-09 20:26:25 seancorfield If it's just going to be identical to -X:deps mvn-pom I wouldn't have thought it was worth the effort? #2020-09-09 20:27:56 alexmiller you were the one here arguing it was breaking people last week... :) #2020-09-09 20:31:20 seancorfield Hahaha... yeah, but you didn't seem very sympathetic to that argument, and I'm not sure having duplicate ways to do the exact same thing is better 🙂 #2020-09-09 20:14:59 alexmiller I had some thoughts on the -Stree thing you ran into though - I think there are several better ways to detect changing deps tree rather than scraping the -Stree stdout. not sure what your requirements on that stuff are but might be useful to talk about #2020-09-09 20:18:47 alexmiller from a clj pov, we're writing edn files (.libs in particular) to the cache dir (can see location with -Sdescribe), so you could be slurping and diffing Clojure maps rather than comparing strings #2020-09-09 20:27:46 seancorfield True, but a) how can you easily get the correct .libs file from a given invocation without scraping the output and b) that gives no sense of depth of transitivity, which is specifically what I rely on in -X:deps tree #2020-09-09 20:28:54 seancorfield "Because reasons"... we go one level down in the full tree (so we consider top-level deps and the first level of transitive deps, but we don't care about lower levels). #2020-09-09 20:30:14 seancorfield We've found that we can safely ignore the 2nd-order transitive deps and if we have a problematic 1st-order transitive dep, we just promote it to the top-level (and then its transitive deps are promoted to 1st-order transitive deps and we start to track them separately). #2020-09-09 20:30:42 alexmiller well the transitivity stuff is all broken in the -Stree output #2020-09-09 20:31:17 alexmiller but maybe you're just not encountering the cases where it's bad (mostly with exclusions) #2020-09-09 20:31:43 seancorfield We don't use exclusions, for the most part. #2020-09-09 20:32:13 seancorfield This is our only exclusion:
com.walmartlabs/lacinia {:mvn/version "0.35.0"
:exclusions [clojure-future-spec/clojure-future-spec]} #2020-09-09 20:32:52 alexmiller the main thing is, the code never actually builds the real expansion tree as a tree. -Stree is kind of faking from the libs map #2020-09-09 20:33:18 seancorfield We decided that managing conflicts via :exclusions was a bit of a nightmare so we control conflicts by promoting transitive deps to the top level. #2020-09-09 20:33:45 seancorfield Right, but I'd have to write that "faking" code myself, whereas -X:deps tree already does a "good enough" job. #2020-09-09 20:34:22 alexmiller well, it exists as a function you could call (the one -Stree calls) :) #2020-09-09 20:34:48 alexmiller but that stuff is due for some significant changes soon as I fix up some of this stuff #2020-09-09 20:34:55 seancorfield Besides, I've had this shell script running with -Stree for ages and don't really want to rewrite it as a Clojure program right now. Maybe when I'm bored and want to go shave some more yaks... #2020-09-09 20:35:47 alexmiller ok, just letting you know that I'm not considering stdout changes from -Stree (or its replacement) to be breaking :) #2020-09-09 20:36:35 alexmiller but I am planning to add some more api fns to get tree-like deps data #2020-09-09 20:19:56 alexmiller and from a tools.deps pov, you could probably make something that did this via the api instead, but not sure if that's worth the effort #2020-09-09 20:21:26 alexmiller I really consider -Stree / -X:deps tree to be human consumer tools and ones likely to hopefully improve over time, changing format. I don't consider the current particular output to be something a script should rely on. #2020-09-10 00:55:53 didibus I'm excited to try the new clj exec, at first glance, its giving me some powershell vibes... which I'm not sure about 😛 #2020-09-10 01:00:25 didibus One question, in the guide it says: clj -A:deps -X:install :jar '"/path/to.jar"', couldn't you include the deps in the :install alias itself? And thus you could then omit the -A:deps and just do: clj -X:install :jar ... #2020-09-10 01:14:25 seancorfield It says clj -X:deps mvn-install :jar '"/path/to.jar"' #2020-09-10 01:14:38 seancorfield Are you looking at the old (stable) guide @didibus? #2020-09-10 01:15:16 seancorfield But, yes, in general you can combine the exec stuff and the dependencies. #2020-09-10 01:38:16 didibus I probably was, it was from the first blog post about it on inside clojure #2020-09-10 01:39:46 didibus Ah yes, the latest blog post says: "so you can supply the deps and the function to execute together" never mind then #2020-09-10 01:42:40 didibus So that's awesome, now all we need is a convention to define alias in some file that you put in a git repo, and something like: clj -X:deps install-alias some-git-repo and a corresponding clj -X:deps remove-alias some-git-repo and we have ourselves a little NPM of some sort. #2020-09-10 01:43:11 didibus Well, it doesn't need to live inside -X:deps, but it would be an easier way to bootsrap it #2020-09-10 01:45:03 didibus You could even do something like: clj -X:deps install-alias :git some-git-repo :symlink true and it could create a symlink in your user /bin to: clj -X:your-alias #2020-09-10 01:45:33 didibus Sorry, none of this has to be included in deps, though I think it could be a good idea #2020-09-10 01:46:08 didibus Then clj could be used to install Clojure command line apps on your machine as well. #2020-09-10 01:47:56 didibus Might need some more hammock time #2020-09-11 14:58:20 souenzzo I'm using clojure 1.10.1.645 on linux
When I do clojure -e "(compile 'app.main)" it results in java .... -e '(compile '\''app.main)' wich works
When I create a alias :aot {:main-opts ["-e" "(compile 'app.main)"]}
It results in java commnad like -e '(compile' ''\''conduit.server)' which will do a EOF error, as it will try to eval (compile
Is it a know issue? I think that I already used it before (it looks like a regression for me)
PS: I'm using bash -x /usr/bin/clojure ... to know which "final java command" clojure is calling #2020-09-11 15:00:28 alexmiller nothing has changed in this area afaik #2020-09-11 15:01:17 alexmiller things in :main-opts go into a file, and then back out of a file which causes problems due to word-splitting in bash #2020-09-11 15:01:36 alexmiller so if you replace the space after compile with a , that will help #2020-09-11 15:01:54 alexmiller this is a known issue (but it has always been a known issue :) #2020-09-11 15:30:17 souenzzo Should I try to PR?
I worked with bash for some years 🙂 #2020-09-11 15:30:41 alexmiller there is a ticket and about 4 different patches on there already :) #2020-09-11 15:31:02 alexmiller let me find it #2020-09-11 15:35:03 alexmiller similar but different issues on windows #2020-09-11 17:17:47 seancorfield I just routinely use , instead of whitespace when writing stuff in deps.edn and even on the command-line for clojure 🙂 #2020-09-11 17:20:58 lread aka the “corfield comma”! #2020-09-11 17:23:16 seancorfield I can think of worse things to be known for 🙂 #2020-09-11 17:51:36 souenzzo Now thinking about Windows.
I can't implement a bash-friendly solution, once it may be dificult to implement in "cmd", once it may need to "interpret" that file as bash.
How is going that idea of do like "deps.exe" and use a graal-binary to replace bash/bat? #2020-09-11 18:22:29 seancorfield Or use WSL2 for Clojure development? 🙂 #2020-09-11 18:49:45 lread If you want to work on plain old Windows and don’t need to stay with official Clojure tools, you might consider trying https://github.com/borkdude/deps.clj. Using its deps.exe in place of clojujre.exe made sense for me when testing my project under Windows on GitHub Actions. #2020-09-11 17:18:24 alexmiller now of course, you can just use -X in many cases instead :) #2020-09-11 17:22:18 seancorfield Although you can't use -X to do a simple compile because it requires a symbol and -X passes a hash map 🙂 #2020-09-11 17:29:00 dominicm I reckon we'll see a pack of map versions of core functions. #2020-09-11 17:48:11 vlaaad All you need is a single invoke :
(defn invoke [{:keys [fn args]}]
(apply fn args))
(invoke {:fn inc :args [1]}) => 2
#2020-09-11 17:29:18 dominicm Or maybe the work on kwargs, will include positional arguments too? #2020-09-11 18:49:04 alexmiller A new prerelease version of clj is now available (1.10.1.681):
• Reinstate -R, -C, and -Spom from last week's prerelease (but still deprecated)
• TDEPS-155 - Some error message improvements for various bad coordinate cases #2020-09-11 18:49:55 alexmiller Based on feedback from last week, added some things back to ease the migration. #2020-09-11 18:52:36 alexmiller For the moment -Spom will not warn. The -X:deps mvn-pom version was really missing support for modifications to the classpath previously pulled from clj invocation itself. I've added that as an arg but needs more thinking. #2020-09-11 18:54:18 alexmiller @seancorfield I've also standardized all the deprecation warnings to be stderr, if you wanted to pipe redirect them to /dev/null and ignore that's at least possible now. also I fixed the exec.jar issue in the installation. #2020-09-11 19:24:35 seancorfield Thanks @alexmiller -- just to confirm, is -R exactly the same as before? It hasn't expanded to support other types of arg maps like -A? #2020-09-11 19:38:14 alexmiller yes, just re-added, not going to do the expanded thing #2020-09-11 20:56:03 markbastian I had an uberjar alias using depstar that was configured with main opts of:
["-m" "hf.depstar.uberjar"
"target/myproject-with-dependencies.jar"
"--compile" "*" ;Note - the * seems to be needed for older versions of the args, but not now.
"--verbose"
"-m" "myproject.main"]
It was using the RELEASE version of depstar. I noticed that now instead of putting the jar in target/myproject-with-dependencies.jar it creates a jar named (Note: Do not ".
Is this the right way to build the standalone jar with aot compilation? It looks like --compile will do transitive compilation from your entry point. Is there a way to specify aoting the whole project? I saw that you can do :aot true. If I did that would it go in the main args or next to the :main-args key in the deps file? #2020-09-11 21:05:36 markbastian Actually, looks like :aot is part of the -X functionality. I missed that. #2020-09-11 21:21:24 seancorfield @markbastian There's a #depstar channel now for dedicated help... #2020-09-11 21:23:12 markbastian Thanks, I should have thought of that. #2020-09-12 02:32:36 hugod With cli 673, and 681
clojure -Stree => Option changed, use: clj -X:deps tree
clojure -X:deps tree => Unqualified function can't be resolved: tree #2020-09-12 03:15:18 seancorfield @hugod It works fine for me (aside from showing tools.deps.alpha as a dependency and everything it depends on -- which was the problem with -X:deps mvn-pom in the last release). #2020-09-12 03:15:41 seancorfield Do you perhaps have your own :deps alias, shadowing the system installed one? #2020-09-12 10:56:28 hugod That was it thanks. Should have thought of that. Might be nice to have a warning if you define your own :deps alias. #2020-09-12 03:16:48 seancorfield (! 993)-> clojure -Sdescribe
{:version "1.10.1.681"
:config-files ["/usr/local/Cellar/
^ @alexmiller FYI #2020-09-12 03:29:28 sogaiu here i get:
$ clj -X:deps tree
Cloning:
Checking out: at e160f184f051f120014244679831a9bccb37c9de
Unqualified function can't be resolved: tree
$ clj -h | grep 681
Version: 1.10.1.681
#2020-09-12 03:31:45 sogaiu looks like i've got the following in my ~/.clojure/deps.edn:
:deps
{
:extra-deps {org.clojure/tools.deps.alpha
{:git/url ""
:sha "e160f184f051f120014244679831a9bccb37c9de"}}
} #2020-09-12 03:33:42 sogaiu ...and with -Srepro i don't get that message #2020-09-12 03:57:13 seancorfield Yup, bad idea. You should only have that under an alias. #2020-09-12 03:58:42 seancorfield You're essentially breaking the :deps alias in the system installed deps.edn... #2020-09-12 03:58:48 seancorfield @sogaiu ^ #2020-09-12 05:38:28 dominicm Maybe the system deps should have a namespace? #2020-09-12 06:05:50 seancorfield Or maybe system aliases should be "reserved" so you can't overwrite them? #2020-09-12 06:19:07 seancorfield Yeah, I changed that when I realized that it conflicted with the system one 😞 #2020-09-12 06:20:06 seancorfield (and that's an outdated branch -- the latest versions of add-lib require builds from source because they include Java code) #2020-09-12 06:20:12 sogaiu ah thanks! #2020-09-12 06:21:40 seancorfield I update that file a lot -- so it's a good idea to keep git pulling it regularly 🙂 #2020-09-12 06:22:15 seancorfield Both -M:new and -X:new will work. #2020-09-12 06:23:00 seancorfield clojure -M:new lib myname/myproject or clojure -X:new :name myname/myproject (since :template is defaulted to lib). #2020-09-13 13:35:53 dominicm I suppose that the new -X stuff can't express regex directly, as it's EDN based. I only mention it because it's one of a handful of types that couldn't be passed to a function directly, and a wrapping string-based function would be needed for use from the CLI. #2020-09-13 13:49:47 alexmiller Yep #2020-09-13 17:35:11 seancorfield Yup, that's what I did for depstar's --exclude arg, but the command-line version is also a plain string so both logic paths have to turn it into a pattern. #2020-09-13 18:26:21 borkdude same with EDN config, I also use strings as regex in clj-kondo's config in places #2020-09-13 18:32:16 kingcode I’m wondering about this strange syntax in <install-dir>/deps.edn:
{…
{:aliases ...}
R
...
}
So it looks like an embedded deps map is used as a key pointing at the -R option? How does this work? I couldn’t find this documented anywhere on the clojure site’s Deps and CLI reference. (Using clojure 1.10.1.590) #2020-09-13 20:32:54 seancorfield @kingcode I don't see that in any of the versions I have installed -- is it possible you've accidentally edited it? #2020-09-13 20:35:43 seancorfield Here's what's in my system install deps.edn for that version:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "8efdebefe0edcecacbddc5dac1dea3bdbec7cdcfb9b8"}, :content ("[email protected]")} #2020-09-13 20:40:35 kingcode @seancorfield Sorry, I meant my config dir, not the install.. I think maybe installing rebl recently caused a bad edit - here is the effective content of .clojure/deps.edn:
{ ;...
{:aliases
{:rebl ;; for JDK 11+
{:extra-deps {com.cognitect/rebl {:mvn/version "0.9.241"}
org.openjfx/javafx-fxml {:mvn/version "15-ea+6"}
org.openjfx/javafx-controls {:mvn/version "15-ea+6"}
org.openjfx/javafx-swing {:mvn/version "15-ea+6"}
org.openjfx/javafx-base {:mvn/version "15-ea+6"}
org.openjfx/javafx-web {:mvn/version "15-ea+6"}}
:main-opts ["-m" "cognitect.rebl"]}
:rebl-jdk8 ;; for JDK 8
{:extra-deps {com.cognitect/rebl {:mvn/version "0.9.241"}}
:main-opts ["-m" "cognitect.rebl"]}}}
R
;;....
} #2020-09-13 20:41:16 seancorfield This is definitely not legal syntax #2020-09-13 20:41:28 alexmiller Agreed :) #2020-09-13 20:42:16 kingcode OK…<sigh-of-relief> thx for your comments. I probably should remove the R and make the embedded map the entire effective content then? #2020-09-13 20:45:08 seancorfield But :aliases should be a top-level key in the overall map. #2020-09-13 20:46:51 kingcode Thanks @seancornfield - that makes sense 🙂 #2020-09-13 21:19:05 dpsutton Aliases always go in a map with the other aliases. That example is correct. And your deps.edn locally will largely look the same yes? #2020-09-13 21:28:38 kingcode woops @dpsutton and @alexmiller Sorry indeed my mistake. The Rebl distro alias example works fine 🙂 Too much staring. #2020-09-14 09:59:16 kirill.salykin UPD: nvm, wrongly specified the maven repo, sorry
---
Hi
I am trying to install dependency from the s3 but facing the issue:
Error building classpath. Failed to read artifact descriptor for com.amazon.redshift:redshift-jdbc42:jar:1.2.47
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.amazon.redshift:redshift-jdbc42:jar:1.2.47
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:255)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
(Following this doc https://docs.aws.amazon.com/redshift/latest/mgmt/configure-jdbc-connection.html#configure-jdbc-connection-with-maven)
{:paths ["src" "resources"]
:mvn/repos {"redshift" ""}
:deps{
com.amazon.redshift/redshift-jdbc42 {:mvn/version "1.2.47"}
...
}}
Please help what I am doing wrong? #2020-09-14 11:57:48 practicalli @alexmiller Will the changes in the pre-release be released as version 1.10.2, or something similar such as 1.10.2-alpha or 1.10.2-RC ? This would make it very easy to document usage of the tooling, as an easy distinction can be made between 1.10.2 and pre-1.10.2 versions. Thank you #2020-09-14 12:34:25 alexmiller No, probably not #2020-09-14 12:40:57 practicalli Ah, so I'll just drop the older content then and add hints to require the latest version is running. If issues arise the the hint will be first to update the version of CLI tools. Thanks #2020-09-14 17:44:05 unbalanced struggling with a proxy just trying to get clj -r going #2020-09-14 17:45:09 unbalanced I've tried setting the settings in ~/.m2/settings.xml and running
clj -Dhttp.proxyHost=<host> -Dhttp.proxyPort=<port> -Dhttps.proxyHost=<host> -Dhttps.proxyPort=<port> -r
#2020-09-14 17:45:22 unbalanced any suggestions? #2020-09-14 18:08:01 borkdude @goomba You need to pass Java opts using -J-D #2020-09-14 18:14:06 unbalanced ahhhhh #2020-09-14 18:14:13 unbalanced 🙏 #2020-09-15 17:43:14 Aaron Cummings Does there exist a tool like Mr. Anderson (dependency shading) for tools-deps? #2020-09-15 17:43:14 Aaron Cummings Does there exist a tool like Mr. Anderson (dependency shading) for tools-deps? #2020-09-15 17:52:23 andy.fingerhut I haven't actually used Mr. Anderson, but am aware of it and why someone might want to use it. Is Mr. Anderson in any way currently tied to Leiningen? #2020-09-15 18:08:19 Aaron Cummings It is a leiningen plugin, but it looks like it could be used directly. But I'm having trouble describing what I'm trying to accomplish, which maybe means I'm reaching for the wrong thing. I'll need to think about this some more. #2020-09-15 18:28:39 andy.fingerhut I have never done this before, but the Mr. Anderson project README has a brief note and a pointer to this repository, which seems to use Mr. Anderson in a way that doesn't require Leiningen: https://github.com/Olical/conjure-deps #2020-09-15 23:07:00 sogaiu it does use it as a library there #2020-09-16 00:41:55 Aaron Cummings I'll take a look at that, thanks. #2020-09-15 20:36:09 rberger I am wondering why I’m getting
shadow-cljs - starting via "clojure"
Cloning:
(Full stacktrace at https://gist.github.com/rberger/34d72e06736ed7cb449c88030c9815ba)
This happens if I have the following in my .ssh/config:
Host
User git
Port 22
HostName
IdentityFile ~/.ssh/github.rsa
AddKeysToAgent yes
TCPKeepAlive yes
IdentitiesOnly yes
Host *
User myname
Port 22
IdentityFile ~/.ssh/id_rsa
AddKeysToAgent yes
TCPKeepAlive yes
IdentitiesOnly yes
All my regular git activity works fine with this, and it seems I need this for macOS automatic key loading to work
If I remove the Host * entry the clojure tools git loading works fine.
I presume this is a jgit issue but I’m wondering if there is a way to make this work as I’m expecting or if I’m expecting wrong.
The Maven issues talk about setting to useNativeGit=true but I don’t know if there is a way to do it with the clojure tools
I had this problem once before and removed the offending config stuff, but I would like to figure out how to make this work with this or equivalent ssh config if possible. #2020-09-15 20:50:35 seancorfield IdentityFile is the problem I believe, based on discussions I've seen here... #2020-09-15 20:51:09 seancorfield But @alexmiller can give you a more authoritative answer... #2020-09-15 20:53:58 alexmiller that's definitely where I'd start (by removing that) #2020-09-15 20:56:09 alexmiller since it looks like you are using a different github key, I'm not sure of the right combination of things to make it work. you might need to add that key with ssh-add #2020-09-16 11:52:14 practicalli https://github.com/practicalli/clojure-deps-edn/tree/qualified-alias-keywords-and-new-flags is a refactor of the project wide aliases in light of the changes coming in Clojure CLI tools (version 1.10.1.681 onwards). This will go live when the Clojure CLI changes go live.
The aliases are using the -X flag for executing functions and providing default hash-map arguments (where the library supports this) and using -M flag for clojure-main (replacing -A and -R). Where chained aliases in a command have multiple main namespaces, the alias that should have its main run is added last in the chain.
Alias namespaces have been upgraded to be qualified names, further defining their purpose and making them easy to understand without looking at the details (at least that is the intent).
https://gist.github.com/jr0cket/ae7dd745eb45870109ace59fe835ce80 contains notes to be added to the Practicalli Clojure book on using the Clojure CLI tools with the recent changes. Any feedback on this is welcome, especially on how clear an accurate this is.
Thank you. #2020-09-16 13:52:22 alexmiller -P can be used with -X too #2020-09-16 13:55:58 alexmiller re "The order in which repositories are consulted is not guaranteed", recently I changed this to actually guarantee that the first two repos consulted are central and clojars (no guarantee after that). central and clojars are listed in the root deps.edn so do not need to be added to deps.edn or settings.xml unless you add mirrors etc. #2020-09-16 13:56:16 alexmiller "`business-area**" looks like bad markup #2020-09-16 13:58:26 alexmiller the automatically included "central" and "clojars" repos can be removed from consideration by nil'ing them in deps.edn {:mvn/repos {"central" nil}} #2020-09-16 17:08:08 seancorfield curl -O
chmod +x linux-install-1.10.1.681.sh
sudo ./linux-install-1.10.1.561.sh
Version is not correct in that last line. #2020-09-16 17:09:51 seancorfield Run project with or without an alias:
`clojure -M:alias -m domain.app-name`
`clojure -M -m domain.app-name
`
Looks like you have extra backticks in that code block. #2020-09-16 17:10:44 seancorfield :project/outdated-mvn - check for newer dependencies (maven only)
The other two lines above this have the alias as code (with backticks) but this third one does not. #2020-09-16 17:12:59 seancorfield clojure -M:inspect/portal-cli ClojureScript node.js REPL with Portal dependency
That should be :inspect/portal-node #2020-09-16 17:15:21 seancorfield Add Reveal as tap> source
I thought Reveal did that automatically? I don't recall having to do this manually when I've used Reveal. Depends on how you start it. If you start it with vlaaad.reveal/repl the add-tap is automatic. Perhaps make it clearer that this section only applies if you have started the REPL some other way than via Reveal itself? Or maybe it was just my misreading of what's being discussed at this point in the README? #2020-09-16 17:15:33 seancorfield "DEPRICATED" => "DEPRECATED" #2020-09-16 17:15:57 seancorfield I found several other aliases further down the README that are not highlighted as code with backticks. #2020-09-16 18:52:36 practicalli Thank you Alex, Sean, most helpful and much appreciated. #2020-09-16 21:02:27 lread Very cool @U05254DQM! Not sure if anyone mentioned what looks like a typo:
* deps = `:deps`, `:extra-deps`, `replace-paths`
* path = `:path`, `:extra-paths`, `replace-deps
`
Last items seem reversed? #2020-09-16 21:59:35 practicalli Good catch. @UE21H2HHD , I'll fix that copy past error, thanks. #2020-09-16 23:30:54 seancorfield Posting here because it there's a larger audience who might have more insight into Maven and its shade plugin behavior: The next version of depstar will preserve license/copyright files coming from dependencies by concatenating them (but omitting exact duplicates). As far as I can tell from inspection of several "uber" JAR files I found in the wild, this is the behavior of Maven with the shade plugin, with the additional license resource transformer (which omits duplicates). I couldn't find anything in Maven's documentation about this behavior, and looking at the source of the shade plugin didn't help much. If anyone here has deep insight into that aspect of Maven and can confirm its actual behavior, that would be very helpful (otherwise I'll just run with the depstar behavior I described). #2020-09-16 23:39:20 alexmiller I've used the shade stuff several times and I'm not sure it is anything mandated by maven either way, you're in making stuff up territory :) #2020-09-16 23:40:18 alexmiller I actually have come to think that the "right" way to do this is store things like that in lib-specific sub dirs under META-INF #2020-09-16 23:40:51 alexmiller I don't think concatenating is wrong though #2020-09-16 23:55:04 seancorfield Yeah, my inspection of various "uber" JAR files seemed to confirm that if maven shade finds META-INF/LICENSE or META-INF/LICENSE.txt in multiple dependencies, it simply concatenates them together in the resulting JAR file. My reading of the Apache license resource transformer seemed to confirm that it attempts to avoid duplicating the text of any license. #2020-09-16 23:55:42 seancorfield But this is definitely all "undocumented territory" as far as I can tell. #2020-09-17 01:03:16 pmonks I wonder if there are legal implications when concatenating license texts? #2020-09-17 01:07:28 alexmiller seems dubious to me :) #2020-09-17 14:13:27 markbastian When I run clj -Spom with an existing pom that already has a <build> tag the generated <build> tag is then placed inside of the existing rather than merging the contents or just adding the <sourceDirectory>. Is this a bug or am I just doing it wrong?
To be clear, it does this:
<-- Contents above elided -->
</plugins>
<build>
<sourceDirectory>src/main/clojure</sourceDirectory>
</build>
</build>
</project> #2020-09-17 14:17:06 alexmiller what version are you on? #2020-09-17 14:17:27 alexmiller there have been bugs and fixes on this #2020-09-17 14:17:42 alexmiller clj -Sdescribe #2020-09-17 14:23:31 markbastian "1.10.1.561" #2020-09-17 14:31:14 alexmiller yeah, that's the newest stable but the fix is not included in a newer stable version yet (it's fixed in the prereleases but you'll get a bunch of other changes with that) #2020-09-17 14:33:05 alexmiller so you could move to a prerelease (1.10.1.681 is newest) or you could fall back to a prior stable version - 1.10.1.510 or earlier #2020-09-17 14:33:41 alexmiller that version was actually also broken but only for -Spom generating a new pom #2020-09-17 14:34:11 alexmiller or you can wait till the next stable version, hopefully coming within the next week #2020-09-17 14:34:46 borkdude or generate the pom with the newest unstable and then move back to the newest stable :) #2020-09-17 14:36:38 borkdude it's pretty easy to do this if you download the clojure shell script for a given release #2020-09-17 14:38:10 alexmiller I prefer "prerelease" to "unstable" :) #2020-09-17 14:45:51 borkdude @markbastian if you're on mac, this is how you can switch versions easily:
brew install clojure/tools/
#2020-09-17 14:46:18 borkdude or if you're using linuxbrew, same #2020-09-17 15:10:09 markbastian Cool, thanks for the help. Waiting for next week is great. I don't need to update the pom much. #2020-09-18 08:53:25 Saikyun is there a way to get clj working on a machine without access to internet? specifically I'd want clj + project dependencies on a machine behind a very restrictive firewall. I'm able to transfer files to it, and when building I use uperdeps. #2020-09-18 08:53:53 Saikyun I know that with leiningen I could download a jar-file and a bat-file, then it didn't need access to internet. is it possible to do something like that with clj? #2020-09-18 09:06:08 dominicm @saikyun the installer just downloads a couple jar files and puts them on the filesystem, then puts a script in /bin/ to run that jar. #2020-09-18 09:45:20 Saikyun thanks for the example 🙂 #2020-09-18 09:07:21 dominicm One idea might be to run the installer into a directory with the -p flag. #2020-09-18 09:07:22 Saikyun the target computer is on windows btw :x #2020-09-18 09:18:44 borkdude @saikyun Alternative if you end up running into issues: with deps.clj it's just a matter of downloading the executable and placing the tools jar in ~/.deps.clj (as documented here: https://github.com/borkdude/deps.clj/issues/20) #2020-09-18 09:24:07 Saikyun @borkdude thanks, something like that is what I wanted 🙂 what is the analogue to ~ on windows? #2020-09-18 09:24:27 borkdude %USERPROFILE% #2020-09-18 09:24:30 Saikyun thanks #2020-09-18 09:34:31 Saikyun cool, thanks #2020-09-18 09:35:04 Saikyun where do you get deps.exe from? #2020-09-18 09:36:28 borkdude or install via scoop etc, but since you're having firewall issues, Github releases is probably the easiest #2020-09-18 09:37:08 Saikyun yeah, thanks a lot. #2020-09-18 09:37:39 Saikyun and it's enough for deps.exe to be on PATH, right? #2020-09-18 09:38:09 borkdude yes #2020-09-18 09:38:26 borkdude also works from cmd.exe, no PS needed #2020-09-18 09:38:39 Saikyun okay, cool 🙂 #2020-09-18 09:38:48 Saikyun I'm getting this error now:
Error building classpath. Failed to read artifact descriptor for org.clojure:clojure:jar:1.10.1
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.clojure:clojure:ja
r:1.10.1
#2020-09-18 09:39:03 Saikyun this is what prompted me to try to install tools.deps in an alternative way in the first place #2020-09-18 09:39:27 Saikyun I wonder if there's some old stuff lying around, trying to find tools.deps in the directory specified when I ran the .ps1 installation script... #2020-09-18 09:39:49 borkdude probably there's something wrong with your .m2 stuffs? #2020-09-18 09:40:22 Saikyun oh, right, I guess I need to put clojure in the .m2 folder? (e.g. by using -p...?) #2020-09-18 09:41:17 borkdude tools.deps is going to the internets when the dep is not yet in .m2 #2020-09-18 09:41:27 Saikyun okay, that makes sense #2020-09-18 10:14:32 Saikyun when running clj -P , where do the dependencies end up? I'm looking in ~/.m2 but can't find clojure :S #2020-09-18 10:14:44 Saikyun I have clojure as an explicit dependency in deps.edn as well #2020-09-18 10:15:37 Saikyun nvm, it was in .m2/org/clojure.. , not .m2/clojure... #2020-09-18 12:01:40 Saikyun got it all working by removing my local, existing .m2, then ran clj -P, then I copied my new local .m2 to the firewalled machine and now it works 🙂 #2020-09-18 12:01:44 Saikyun thanks a lot for the help #2020-09-18 12:21:10 borkdude I think the .m2/repository is also configurable in deps.edn (can't remember it from the top of my head) #2020-09-18 12:40:33 alexmiller Yes, with :mvn/local-repo #2020-09-18 12:47:39 borkdude which probably also works with -Sdeps? then that could be used for populating an ad hoc repo @saikyun #2020-09-18 12:49:01 alexmiller Yep #2020-09-18 14:10:38 alexmiller I use this for testing "clean cache" cases all the time #2020-09-18 14:10:55 alexmiller (without having to muck with my actual ~/.m2/repository) #2020-09-22 06:57:26 Saikyun ahh, thanks for the tip! great to know 🙂 #2020-09-22 06:57:46 Saikyun I’m setting up a dev environment on a pretty locked down production machine, so these things are invaluable #2020-09-22 07:00:27 borkdude One issue I had while developing on a train is that I forgot to download deps for certain aliases #2020-09-22 07:01:33 Saikyun ah, good point. will keep that in mind #2020-09-18 19:09:43 just.sultanov Hi all! Does clojure take MAVEN_OPTS from environment variables? I can’t use keystore with clojure, but it’s work with mvn (directly) #2020-09-18 19:14:26 seancorfield @just.sultanov I just grepped the source of brew-install, homebrew-tools, and tools.deps.alpha and didn't see that var in any files so I would say "no". #2020-09-18 19:15:51 seancorfield It does handle ~/.m2/settings.xml for Maven stuff tho' so maybe you could put credentials in there? #2020-09-18 19:24:03 just.sultanov I need to use keystore.jks with self-signed corp certificates. I don’t have problems with credentials. Maven by default pick up settings from .mavenrc and MAVEN_OPTS, but I didn’t see any usages in sources of tools.deps #2020-09-18 19:44:31 alexmiller doesn't use it right now #2020-09-21 14:43:44 timo What is the command to only resolve dependencies with tools-deps and not start a repl? I need it for CI. #2020-09-21 14:44:46 alexmiller clj -e nil is sufficient. current prereleases have a new -P (prepare) switch for this purpose. #2020-09-21 14:46:47 timo thanks! #2020-09-21 15:03:52 pmonks For better logging in multi-step builds, I sometimes do clojure -Srepro -e '(println "Dependencies downloaded")' . #2020-09-21 14:47:35 ghadi @timok you'll want clojure -e nil instead of clj if you don't have an interactive terminal #2020-09-21 14:47:42 ghadi in CI #2020-09-21 14:47:55 timo ok, cool #2020-09-21 14:48:09 timo I never understood the differences though #2020-09-21 14:48:43 dominicm Clj calls rlwrap. So adds things like up/down. #2020-09-21 14:49:48 alexmiller right, sorry #2020-09-21 14:50:11 timo awesome, thanks for the answers 👍 #2020-09-22 02:26:10 alexmiller clj 1.10.1.693 (prerelease) is now available:
• Classpath ordering changes. Classpath will always now be ordered extra-paths, paths, then libs. Libs are ordered by tree depth, then alpha. These changes both revert some ordering changes in the last stable release and go further in defining a reproducible classpath order.
• clj -Stree re-enabled for now while we decide on best way to convey classpath-changing options to -Stree and -Spom
• Windows scripts updated to latest
• This prerelease version should be considered a release candidate for the next stable version! If you try it and have feedback, please leave it here #2020-09-22 03:20:26 seancorfield @alexmiller I assume the -Stree restoration is because of this difference?
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "a3d0c6c2cdc0e3e7e6f0e8f7ecf38ef2f691f6e992ed"}, :content ("[email protected]")} #2020-09-22 04:02:46 alexmiller yes, but that's just a symptom of the greater problem, which is how to convey classpath changing arguments to tools like pom and tree without doing it through clj's args #2020-09-22 04:03:18 alexmiller Datomic's ion-dev has the same need (for lib bundling) #2020-09-22 04:03:44 alexmiller so we're looking at something common for all these but don't want to gate on figuring that out #2020-09-22 04:10:29 alexmiller The -X:deps tree stuff is going to have more features over time for other variants of viewing the tree etc #2020-09-22 05:19:23 seancorfield Interesting. Thanks, Alex. #2020-09-22 10:59:32 dominicm I thought that the corfield comma was documented on https://clojure.org/guides/faq but it seems not. Is it documented on the site somewhere? It would be useful for the http://nrepl.org docs to link to as it is tripping users up. #2020-09-22 19:17:09 ghadi how do people handle bumping the version inside pom.xml with minimal effort? #2020-09-22 19:17:36 ghadi i'm not beneath sed but I was hoping for something intelligent #2020-09-22 19:22:19 alexmiller I've done manual, various Maven plugins, sed #2020-09-22 19:22:49 alexmiller the contrib projects actually have this done automatically via Jenkins Maven integration #2020-09-22 19:29:14 seancorfield I edit it manually as the last step of preparing for a release -- and I edit both the <version> and the <tag> entries to be consistent (the latter just has v at the beginning). I tend to do a global find for a regex that captures the current release, the new (planned) release, and my generic placeholder (e.g., 1.1.next) which I may tag new functionality with in the readme, docs, code, and change log. Then I just work my way through those find results one at a time, and I can eyeball that all matches correctly refer to the new release and the old release as needed. #2020-09-22 19:30:04 seancorfield (so I might search for 1\.\1.(next|588|623) for example) #2020-09-22 19:55:57 vlaaad Ugh, why not just create pom.xml from some tdeps-generated inputs? #2020-09-22 19:57:04 vlaaad Bumping the version will be a different arg to a pom-generating fn #2020-09-22 20:00:45 borkdude @ghadi clojure.data.xml is included in bb, I've done XML rewriting with that #2020-09-22 20:06:33 hugod makejack updates the pom based on a :version tag in a project.edn file. I haven’t got a task to bump the version yet. #2020-09-22 20:07:34 ghadi sed it is #2020-09-22 20:09:38 alexmiller in case it saves you any time... (made for Mac sed) #2020-09-23 14:09:21 lread In the past I have also used maven to do this type of work, for example to update the tag and version:
mvn versions:set-scm-tag -DnewTag=$(git rev-parse HEAD) -DgenerateBackupPoms=false
mvn versions:set -DnewVersion=$(bin/generate_version) -DgenerateBackupPoms=false
Somehow it bothers me how long it takes maven to do this tho… #2020-09-23 15:39:08 alexmiller anyone tried the clj prerelease yet? #2020-09-23 16:05:25 vlaaad I haven’t, but I’d like to do it today #2020-09-23 16:25:38 alexmiller 👍 #2020-09-23 16:33:10 seancorfield Installed it Monday night and integrated it into our dev/test/build pipeline -- everything is working as expected. REPLs, aliases, running tests, building uberjars... #2020-09-23 16:53:43 alexmiller Thx, good to hear #2020-09-23 18:44:13 vlaaad I'm just trying to understand the basis stuff, is this intended that :parents of a thing contains a thing itself?
cljfx/cljfx$jdk11
{:mvn/version "1.7.9"
:deps/manifest :mvn
:dependents [cljfx/cljfx$jdk11 cljfx/cljfx]
:parents #{[cljfx/cljfx cljfx/cljfx$jdk11]
[cljfx/cljfx]}
:paths ["C:\\Users\\Vlaaad\\.m2\\repository\\cljfx\\cljfx\\1.7.9\\cljfx-1.7.9-jdk11.jar"]} #2020-09-23 18:44:41 vlaaad (that's an entry in :libs) #2020-09-23 18:56:31 alexmiller you should consider :parents to be undefined for now #2020-09-23 18:57:16 alexmiller in other words, that's internal implementation stuff leaking out and I make no guarantee it will be there or be anything in particular :) #2020-09-23 18:58:07 vlaaad ah, okay. what's not internal there? #2020-09-23 18:58:17 alexmiller everything else #2020-09-23 18:59:30 alexmiller to your original question, I'm not sure why'd you see that though, if you could send me your deps.edn, I'd like to look at it #2020-09-23 19:24:13 vlaaad (clj on windows, if that matters) #2020-09-23 19:46:50 alexmiller doesn't matter. I look at it's a quirk of how classifier libs work in maven. when you have a lib foo and a classifier jar foo$a they technically both use the same pom. in this case, the root jar depends on the classifier jar, which has the same pom, so it looks like it depends on itself. #2020-09-23 19:47:03 alexmiller so, expected but weird :) #2020-09-23 19:53:04 vlaaad ah, I see, thanks for clarification! #2020-09-23 16:52:10 ghadi works beautifully @delaguardo. Thanks so much! #2020-09-23 17:01:37 seancorfield @delaguardo I'm using your setup for GH Actions on several of my projects so "Thank you!" -- one comment: using tools-deps for the CLI version is a bit confusing since tools.deps is a library with a different version numbering schema -- Clojure CLI 1.10.1.693 uses tools.deps 0.9.799 for example. #2020-09-24 15:38:08 seancorfield That looks good! #2020-09-24 15:53:52 delaguardo merged, thanks again for pointing on that #2020-09-23 21:53:22 alexmiller Hey, sorry I didn't catch your question earlier in the flow... #2020-09-23 21:54:05 alexmiller I am honestly reluctant to fix this the more I think about it and talk to others about it #2020-09-23 21:55:28 alexmiller There are a lot of security concerns around getting someone's else repository, which may either be an internal repo or a repo that provides rogue versions of deps defined in central repos #2020-09-23 21:55:52 alexmiller So my recommendation right now would be to add those repositories explicitly to your deps/pom instead #2020-09-23 22:02:03 grzm Thanks for the feedback! We ran into this today, so I wanted to see what the current thinking on this is. Cheers! #2020-09-23 20:25:11 seancorfield @alexmiller We haven't seen any impact of the changes to classpath order (yay!) but that made me wonder about pom.xml generation: is the order of the <dependencies> section there important, from a Maven p.o.v.? i.e., does it affect the order things show up on the classpath and does clojure -Spom generate the <dependencies> section in a reproducible order now based on the classpath? #2020-09-23 21:01:27 alexmiller good question. the order is important to maven (you should get deps in the order listed in the pom) and I have not explicitly looked at that ordering wrt deps #2020-09-23 21:07:51 vlaaad so that might be unstable if deployed library is used in a project that uses maven as a build tool? #2020-09-23 21:11:07 alexmiller well, deps.edn takes its deps in a map so from a clj pov, there is no order set by the user (the new changes order by alpha) #2020-09-23 21:12:50 alexmiller @seancorfield if you wanted to file a jira about this, I'd be happy to look at it #2020-09-23 22:14:19 seancorfield @alexmiller I don't care enough about the issue to file a Jira ticket. I was mostly just curious if you knew off the top of your head whether -Spom generated <dependencies> in the same order as they would now appear on the classpath. #2020-09-23 22:14:44 seancorfield (I didn't care much about the non-reproducible classpath ordering before either -- although some others did) #2020-09-23 22:17:55 alexmiller I looked at the code and the answer is no - they are not necessarily in the same order they'd appear on the classpath #2020-09-24 05:38:10 andy.fingerhut This may be a distraction from real tools.deps work, but was wondering if others had come across a series of articles by Russ Cox on Golang and package versioning. This is one of those articles I have looked at recently: https://research.swtch.com/vgo-principles. One thing I found interesting is the number of different package managers that use SAT solvers to find what collection of modules and their versions to use. In case you are curious, Russ argues that maybe this is a bad idea. #2020-09-24 13:05:03 alexmiller Yeah I’ve read these. Some of the Go stuff is similar to the tools.deps resolution #2020-09-24 15:00:39 andy.fingerhut I've only read a couple of Russ Cox's articles, but he seems to do a thorough job of writing about pros and cons of various approaches, with examples. #2020-09-25 04:46:24 andy.fingerhut Out of curiosity, did you consider the go module version approach of using the minimum version among all requests for a dependency, rather than the latest, in Clojure CLI tools? It sounded unintuitive at first, but the examples in those articles do seem to make it pretty clear that it is at least one way to achieve better reproducibility in versions of libraries used by an application. #2020-09-25 04:47:54 alexmiller bleh #2020-09-25 04:49:12 alexmiller if you are developing for growth, not breakage, that doesn't make sense #2020-09-25 14:49:03 andy.fingerhut Go has apparently strongly recommended backwards compatibility in APIs and behavior since 2013, before Spec-ulation talk, for similar reasons. I think the "largest of the minimum required versions" of this latest Golang vgo tool is fairly reasonably argued for on the basis of reproducible builds, regardless of whether someone releases a new version of a lib tomorrow. Anyway, not trying to change Clojure CLI tools in this regard. #2020-09-24 06:47:41 vlaaad I like the new exec stuff.
User deps.edn :
{:aliases {:reveal {:extra-deps {vlaaad/reveal {:mvn/version "1.0.130"}}
:ns-default vlaaad.reveal}}}
Starting a repl is as simple as clj -X:reveal repl
❤️ #2020-09-24 06:55:13 sogaiu yes, i liked that too.
was hoping i could stack -X to use multiple tools (like one could with -A), but i get the impression that's not part of the design intent. #2020-09-24 06:56:55 vlaaad I wonder if that can be done using injected basis #2020-09-24 07:03:05 sogaiu haven't thought about that angle yet, may be there's something to that.
i did think one might be able to make one thing that's designed to do the stacking for anything else that might follow its conventions. #2020-09-24 10:37:46 practicalli @vlaaad I would encourage people creating tools for Clojure CLI to support theuse the -X flag. As well as using key value pairs for clarity you can also set default values, making the comand line even simpler
As example, using Sean's clj-new project and the alias below I can create a library project with a default name by just running the command clojure -X:project/new
:project/new
{:extra-deps {seancorfield/clj-new {:mvn/version "1.1.215"}}
:exec-fn clj-new/create
:exec-args {:template lib :name practicalli/playground}
:main-opts ["-m" "clj-new.create"]}
I can then choose to over-ride value in the :exec-args default values by including key value pairs on the command line
clojure -X:project/new :name practicalli/banking-on-clojure #2020-09-24 10:42:22 vlaaad I mean that's exactly what I do :) #2020-09-24 10:46:06 practicalli So I could have an alias similar to this, that would let me do clojure -X:inspect/reveal to run a repl and clojure -X:inspect/reveal :command prepl to run prepl instead
:inspect/reveal
{:extra-deps {vlaaad/reveal {:mvn/version "1.0.130"}}
:exec-fn vlaaad.reveal
:exec-args {:command repl}}
Does that make sense?
To support versions before the pre-release you could also include main-opts
:inspect/reveal
{:extra-deps {vlaaad/reveal {:mvn/version "1.0.130"}}
:exec-fn vlaaad.reveal
:exec-args {:command repl}
:main-opts ["-m" "vlaaad.reveal" "repl"]}
#2020-09-24 11:10:27 vlaaad there is no vlaaad.reveal fn #2020-09-24 11:10:55 vlaaad you can use :exec-fn vlaaad.reveal/repl #2020-09-24 11:13:28 vlaaad my example earlier used ns-default because there are various types of repls that I might want to start, so I want that to be a command line option — it doesn’t prevent :exec-fn use #2020-09-24 11:18:15 vlaaad I don’t see why there should be a single top-level function that receives :command as argument, exec-fn already solves the problem of calling different functions in the ns. Various repls in vlaaad.reveal ns are independent functions with their own different arguments, smashing them together would be unnecessary complecting #2020-09-24 11:42:01 practicalli I was only suggesting an example of a general form of alias, along the lines of the clj-new as that is all I am familar with at the moment. Looking forward to see what develops with reveal. #2020-09-24 12:04:43 sogaiu nice pun 😉 #2020-09-24 12:43:34 vlaaad yeah, I’ll think of a more versatile suggested alias to put into reveal docs once -X is no longer pre-release :+1: #2020-09-24 10:46:41 dominicm A lot of tools support later command flags overriding earlier ones anyway 🙂 #2020-09-24 12:13:51 sogaiu i have something for starting a socket repl on a specific port.
invocation is like:
clj -X:alc.socket-repl :port 9123
there is also a mode where you don't specify a port and it tries to choose one for you:
clj -X:alc.socket-repl
Socket repl port: 38735
Clojure 1.10.1
user=>
here it will print the chosen port number out so it's less digging around you have to do to get this info to your editor or whatever other tooling.
it also makes the port info available programmatically -- jcmd <pid> VM.system_properties should have it show up in the way clojure arranges for it w/ clojure.server.repl=... (i have a separate tool that can obtain this info)
at the end it just hands things off to clojure.main/main with --repl.
i wish i could use this along with other folks' things -- so far -X doesn't look to be composable. what i should say is not necessarily that, but some way to compose things -- it doesn't have to be via -X. #2020-09-24 15:44:46 seancorfield Composability of "main" functions is problematic though: there's no guarantee a "main" will exit (if it starts a server, for example) and aliases in general are merged to produce the final configuration that is used to start a (single) program. #2020-09-24 15:46:56 seancorfield What has been suggested (several times, here in this channel) is to write your own "runner" function that takes a list of functions to execute and runs them one after the other. Now that programs can get at the basis easily (if they are run via the CLI tools -- not if they are run via a JAR!), you could pass a list of aliases and the "runner" could walk that list and exec each alias... #2020-09-24 15:47:30 seancorfield ...you wouldn't get the same semantics of composition but you would get "multiple main" functions run (and you could control the semantics yourself). #2020-09-24 15:48:12 seancorfield (this is exactly why we have a build script that wraps the CLI: you can specific multiple "commands" to build and it runs the CLI for each command, by way of a shorthand) #2020-09-24 22:33:34 alexmiller has anyone tried the clj prerelease 1.10.1.693 on windows? #2020-09-24 22:34:08 alexmiller I've done a fair amount of testing with it, but would be great if someone else had used it #2020-09-24 23:05:25 sogaiu in case others want to try and don't want to look for instructions (https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows):
0) remove previous installation (for me this was deleteing the ClojureTools folder under Documents\WindowsPowerShell\Modules)
1) download: https://download.clojure.org/install/win-install-1.10.1.693.ps1
2) did the "unblock" thing via properties (right-click on file's icon in windows explorer and choose properties)
3) execute the ps script via powershell
4) choose 2 (for under Documents\WindowsPowerShell\Modules) and wait a bit
5) clj -h to verify version
6) freshly clone a clojure project that uses deps.edn
7) start it via clj
something close to those steps worked for me. #2020-09-25 19:00:09 seancorfield @alexmiller Now that 1.10.1.697 is a stable release, any more thoughts on this discrepancy?
(! 740)-> clojure -Sforce -Stree
org.clojure/clojure 1.10.1
org.clojure/spec.alpha 0.2.176
org.clojure/core.specs.alpha 0.2.44
(! 741)-> clojure -Sforce -X:deps tree
org.clojure/clojure 1.10.1
org.clojure/core.specs.alpha 0.2.44
org.clojure/spec.alpha 0.2.176
org.clojure/tools.deps.alpha 0.9.810
javax.inject/javax.inject 1
com.cognitect.aws/api 0.8.408
... lots more stuff ... #2020-09-25 19:03:13 seancorfield (I verified we do not see the same discrepancy with -Spom vs -X:deps mvn-pom) #2020-09-25 19:05:29 alexmiller I'll get to it #2020-09-25 19:05:51 alexmiller have not had a chance to finalize the plan for it yet #2020-09-25 19:06:30 alexmiller for now, -Stree exists #2020-09-25 19:29:28 alexmiller Clojure homebrew tap has the latest of course. Homebrew core and linux core should get this automatically now, but I have no idea how long that will take, looking forward to watching that :) #2020-09-26 14:25:24 sogaiu @borkdude it's a minor thing but doing deps -h gives the version string twice here:
$ deps -h
Version: 1.10.1.697
Version: 1.10.1.697
...
#2020-09-26 14:27:53 borkdude ok, going to fix now #2020-09-26 14:44:54 borkdude @sogaiu Fixed (and all remaining open issues as well). If there's nothing else, going to release #2020-09-26 17:50:23 practicalli I've updated the practicalli/clojure-deps-edn GitHub project and replaced all uses of -A and -R flags for using all aliases, substituting them for -M without any issues... it actually makes things simpler 🙂
Details are in #announcements
Thank you Alex (and Sean) #2020-09-26 19:24:40 dominicm Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate clojure/run/exec__init.class, clojure/run/exec.clj or clojure/run/exec.cljc on classpath.
Full report at:
/tmp/clojure-16205359177864566580.edn
Presumably this means I need to correct something in the void installer, what is that? #2020-09-26 19:25:28 borkdude @dominicm Is this the normal clojure CLI? #2020-09-26 19:26:24 dominicm Ah, a new exec.jar :) #2020-09-26 19:27:09 dominicm OK, made that part of the new void installer :) #2020-09-26 19:27:27 borkdude what's a void installer? #2020-09-26 19:43:52 dominicm Void Linux installer :) #2020-09-26 19:27:56 borkdude @alexmiller Feedback on error message with the new exec:
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "5b393429303f2e3f3e1b16190b696b6a62"}, :content ("[email protected]")}
I think the latter could be improved #2020-09-26 19:36:27 alexmiller not sure I understand why that's failing #2020-09-26 19:36:42 borkdude because there's no val to the key #2020-09-26 19:36:58 borkdude I think #2020-09-26 19:37:27 alexmiller oh, that's probably it #2020-09-26 19:37:27 alexmiller it looks at whether the non-alias arg count is even or odd #2020-09-26 19:37:43 alexmiller there is some fiddly code there trying its best to guess what's wrong #2020-09-26 19:38:05 borkdude can aliases be whitespace delimited? #2020-09-26 19:38:15 alexmiller no #2020-09-26 19:38:30 borkdude then where is the ambiguity? #2020-09-26 19:38:53 alexmiller I don't think there is here, just saying that's the code at faultl #2020-09-26 19:39:02 borkdude ah ok. want me to make a JIRA? #2020-09-26 19:39:31 alexmiller sure #2020-09-26 21:29:07 vlaaad btw, real world example where this might be super confusing: I tried passing var as an argument (`#'inc`), but shells treat # as a start of a comment, so... #2020-09-26 23:59:43 alexmiller Yeah, I’ll fix #2020-09-26 19:40:18 alexmiller there was a period of time where other variants were accepted, this might be a legacy of that, don't remember #2020-09-26 19:40:23 borkdude Unrelated. If checking in binaries into git wasn't an anti-pattern, I think Clojure git deps could be used to implement an ecosystem of binaries more or less like brew, but using deps.edn. The classpath can then be used as the binary path, a bit similar to a nix env. I would have a purpose for this (babashka pods). The git deps stuff could be made to work with GraalVM as well I think, so everything could be very fast after deps have been downloaded (but even without GraalVM, it's already fast due to classpath caching). #2020-09-26 22:19:37 sogaiu i experimented with this once using git lfs. #2020-09-26 22:20:32 sogaiu the binaries in git thing and finding things under ~/.gitlibs that is #2020-09-26 22:22:07 borkdude ah interesting. what did come out of it? #2020-09-26 22:26:35 sogaiu well for one thing i think i learned that not everyone has git lfs 🙂 #2020-09-26 22:26:47 sogaiu though it looks like typical git installs on windows get it #2020-09-26 22:27:05 sogaiu other than that it seemed to work pretty well #2020-09-26 22:28:24 sogaiu in addition to the size aspect of the repository, trying to but binaries for multiple platforms in one repository didn't feel great because the end user then gets extra cruft. #2020-09-26 22:29:23 sogaiu yeah, i saw that page first 🙂
the .edn file inside idea seems preferable to the binaries in the repository idea #2020-09-26 22:29:50 borkdude yeah. also the binaries could be pulled the first time you are trying to find them #2020-09-26 22:30:23 borkdude this idea could even work for non-gitlibs as well #2020-09-26 22:30:41 borkdude maybe there could be a nice coupling with github releases somehow #2020-09-26 22:30:43 sogaiu what would the ultimate location of the binaries be? #2020-09-26 22:31:33 borkdude the resolve function will download the link and the put them inside .foobar/your/dependency/<sha>/the-binary and then that binary file will be invoked #2020-09-26 22:32:13 borkdude where .foobar is some name for this package manager layered on top of tda #2020-09-26 22:36:15 sogaiu ah ok, thanks for the explanation -- i see it at the issue 🙂 #2020-09-26 19:40:47 borkdude Wonder what others think. Maybe instead of checking in the binaries themselves, there could be links to releases in the source, with checksums, etc. But most of the gnarly dependency stuff could be handled by tda. #2020-09-26 19:49:34 borkdude Implementing something like brew this way would only require a couple of tens of lines of code maybe. #2020-09-26 19:58:04 borkdude This could be a package manager for macOS, linux and Windows at the same time #2020-09-27 12:10:26 sogaiu @borkdude for linux, i changed the .cpm.edn file for babashka to be:
{:package/name org.babashka/babashka
:package/description ""
:package/version "0.2.1"
:package/license ""
:package/artifacts
[{:os/name "Mac.*"
:os/arch "x86_64"
:artifact/url ""
:artifact/executables ["bb"]}
{:os/name "Linux.*"
:os/arch "amd64"
:artifact/url ""
:artifact/executables ["bb"]}]}
then i was able to follow your readme. #2020-09-27 12:11:14 borkdude PR welcome :) #2020-09-27 12:11:24 sogaiu lol, ok, i'll send one 🙂 #2020-09-27 12:14:03 borkdude I pushed an update to bb, now cpm is also executable using bb:
$ bb -cp src:test-resources -m cpm.main clj-kondo
/Users/borkdude/.cpm/packages/clj-kondo/clj-kondo/2020.09.09
which is fast :) #2020-09-27 12:21:05 sogaiu indeed:
$ time bb -cp src:test-resources -m cpm.main clj-kondo
/home/user/.cpm/packages/clj-kondo/clj-kondo/2020.09.09
real 0m0.015s
user 0m0.009s
sys 0m0.006s
#2020-09-27 18:54:09 dominicm Void Linux has now updated to clj 697 😃 #2020-09-28 08:49:37 viesti humdum, I sometimes get this error in CirleCI while building project:
clojure -A:... -m kaocha.runner analysis
Picked up JAVA_TOOL_OPTIONS: -Dclojure.main.report=stderr -XX:-OmitStackTraceInFastThrow
Cloning:
using cimg/clojure:1.10.1 as image, with Clojure cli version 1.10.1.536 #2020-09-28 08:50:08 viesti might be totally relared to this project only, but just thinking out loud, that might there be a concurrency issue in downloading dependencies #2020-09-28 13:09:31 alexmiller You could try adding -Sthreads 1 and see if that helps #2020-10-02 08:09:12 viesti thanks! 🙂 #2020-09-28 15:58:09 jjttjj I just updated my clj tools to the latest version and am now getting
CompilerException java.lang.RuntimeException: Unable to resolve symbol: requiring-resolve in this context, compiling:(cljfx/context.clj:37:32)
which would seem to imply that an older version of clojure being used? Is there an obvious reason this would happen after the clj update? (Both cljfx and my project seem to explicitly depend on clojure 1.10.1) #2020-09-28 17:34:37 jjttjj Ok I figured this out. I had a :local/root dependency which points to a standalone jar file which uses clojure 1.9, which seems like it causes the repl started from clj to be 1.9. Is this expected? #2020-09-28 16:04:56 borkdude @jjttjj clojure -Stree is your friend here #2020-09-28 16:05:08 borkdude (or whatever is the way of doing the tree now) #2020-09-28 16:49:15 alexmiller Still that for now #2020-09-28 16:10:40 jjttjj Hmmm it looks like clj is giving me a clojure 1.9 repl even though my project only has 1.10 in the tree. might be a scoop issue going to try reinstalling clj
Edit: ok I think this is a scoop issue #2020-09-28 16:37:57 nate a couple of things I've noticed that are different about the new cli tools:
- Only the first -M is taken. I used to be able to do -A:one -A:two before and both :one and :two were used (was useful for a script I had that wrapped a call to clojure and I could add extra args if I wanted)
- If I put -Spath at the end of my command, it's ignored if there's a -M.... Specifically clojure -Spath -M:repl prints a classpath, but clojure -M:repl -Spath runs the main specified in the :repl alias #2020-09-28 16:50:25 borkdude @nate -M:one:two #2020-09-28 16:50:39 alexmiller -A and -M have always supported multiple aliases so -M:one:two is what you need there. -M and -X should now be last as they grab all args afterwards #2020-09-28 16:51:18 alexmiller So clojure -Spath -M:repl #2020-09-28 16:52:05 nate right, I know that you can put them together, my point is that you used to be able to supply them separately #2020-09-28 16:52:41 alexmiller Now you can’t #2020-09-28 16:52:44 nate I definitely understand why the behavior changed #2020-09-28 16:52:46 nate cool #2020-09-28 16:53:00 alexmiller Multiple -A should still work #2020-09-28 16:53:49 nate interesting point about -A #2020-09-28 16:54:03 nate deprecation warning is pushing me away from that, for good reasons #2020-09-28 16:54:36 borkdude clojure -A:foo -M:bar should also work I think? brings in deps/paths from foo and bar but executes main from bar? #2020-09-28 16:54:44 alexmiller yes #2020-09-28 16:55:26 seancorfield There are a lot more changes in this version than I think are perhaps obvious on the surface -- it's probably always been "best practice" to specify :main-opts separately via a single -M option after using one or more -A options to accumulate aliases but that's never really been codified/written down. This new release sort of "enforces" (or at least "strongly encourages") what I think the core team have always considered best practice... #2020-09-28 16:56:21 nate yeah, I agree with that #2020-09-28 16:56:53 seancorfield -M used to only bring in :main-opts but now brings in paths/deps as well which should not cause any surprises -- but it is quite an important difference. #2020-09-28 16:57:15 borkdude are you referring to a previous pre-release or release? #2020-09-28 16:58:04 seancorfield I'm talking about this in the context of "previous stable" and "current stable" releases. #2020-09-28 16:58:15 nate me too #2020-09-28 16:58:22 borkdude I didn't even know there was an -M in a previous release #2020-09-28 16:59:07 borkdude A minor thing but -M -m still feels weird to me :) #2020-09-28 17:00:10 alexmiller consider it encouragement to use -X :) #2020-09-28 17:03:52 borkdude -X is ok but I don't consider it a substitute for a command line interface for general usage #2020-09-28 17:04:07 borkdude i.e. there will still be a place for tools.cli based tools #2020-09-28 17:04:23 borkdude unless I'm missing something #2020-09-28 17:11:49 borkdude But I guess the -m gets hidden in an alias mostly anyway :) #2020-09-28 17:12:25 tvaughan I was doing something like clojure -A:alias-with-main-opts -m some.other.main With the new release I get a deprecation warning to switch to -M, but with -M -m some.other.main are passed as command-line options to the main entry point associated with alias-with-main-opts. Does this mean there's now no way to override a main entry point? #2020-09-28 17:13:32 seancorfield @tvaughan I believe clojure -A:alias-with-main-opts -M -m some.other.main would be the equivalent now. #2020-09-28 17:14:42 seancorfield (I haven't tried that...) #2020-09-28 17:18:17 tvaughan Thanks @seancorfield. Interestingly, this clojure -A:toolbox:dxf-tools -M -m server.core results in an error message from my application about -m not -M being an unknown option. The main-opts associated with dxf-tools are still being used #2020-09-28 17:19:45 seancorfield Hmm, sorry. Guess I'll have to do some digging (unless @alexmiller can give a quick answer off the top of his head). #2020-09-28 17:20:55 tvaughan No worries. I appreciate the help. I can dig further too. I took a quick look at the changelog but didn't see anything about this specifically #2020-09-28 17:23:10 seancorfield Ah, I think this is one of those cases where the advice is to split up :alias-with-main-opts into two aliases -- one with :main-opts (and any deps needed solely for that) and one with all the (other) deps. #2020-09-28 17:24:30 seancorfield (the other option is to continue using the -R and/or -C options even though they are deprecated) #2020-09-28 17:26:19 tvaughan Cool. Thanks. I thought that might be the case. Didn't want to change things without confirmation tho :) #2020-09-28 17:46:20 alexmiller if you're including an uberjar in your classpath, then you've broken the ability of tools.deps to properly resolve libs #2020-09-28 17:46:44 alexmiller b/c it can't "see" inside the uberjar to tell what libs are included there or their versions #2020-09-28 17:52:30 jjttjj That makes sense #2020-09-28 18:15:00 seancorfield @alexmiller Now that 1.10.1.697 is the stable release, there are four mentions of clj -m <some.ns> in https://clojure.org/guides/deps_and_cli that need to be updated to include -M if you want folks to avoid the clojure.main warnings (i.e., "WARNING: When invoking clojure.main, use -M"). #2020-09-28 20:02:14 alexmiller Yeah it’s on my list to update this week #2020-09-28 18:15:17 seancorfield Those all need to be clj -M -m <some.ns> now, right? #2020-09-28 18:23:50 seancorfield (also, I added a note on TDEPS-100 that indicates some confusion over -A since the reference page does not define clj-opt and clojure -h says that -A is an exec-opt which seems to be at odds with the CLI/deps reference text) #2020-09-28 19:21:20 borkdude yes, now I feel the issue we discussed last weekend. I have this:
{;;:paths ["your-packages"]
;;:deps {your-org/your-packages {,,,}}
:aliases
{:glam {:extra-deps
{borkdude/glam {:git/url ""
:sha "4599fb019deae9418d76a9996ae19b4003f3cc96"}}
:main-opts ["-m" "glam.main"]}}}
and now I want to do:
bb -cp $(clojure -R:glam -Spath) -m glam.main --install clj-kondo/clj-kondo --global --verbose
but I have to make two aliases now, so from Clojure this becomes:
clojure -M:glam-deps:glam
:/ #2020-09-28 19:22:23 borkdude with -Spath it's pretty clear you don't want to run the main right #2020-09-28 19:22:54 borkdude so why not ignore the main when doing this:
clojure -A:glam -Spath
#2020-09-28 19:23:36 borkdude that doesn't seem to be breaking #2020-09-28 19:33:04 borkdude ah well, bash to the rescue:
{:aliases
{:glam/packages {:extra-deps
{borkdude/glam {:git/url ""
:sha "4599fb019deae9418d76a9996ae19b4003f3cc96"}
;; your-org/your-packages {,,,}
}
;; :extra-paths ["your-packages"]
}
:glam {:main-opts ["-m" "glam.main"]}}}
alias glam='clojure -M:glam/packages:glam'
$ glam --install clj-kondo/clj-kondo org.babashka/babashka
#2020-09-28 20:03:45 seancorfield @borkdude If you put -Spath first it will not run the main. #2020-09-28 20:04:13 borkdude ah really #2020-09-28 20:04:24 borkdude thanks! #2020-09-28 20:05:08 seancorfield And that's supposed to be clear from the docs which show clj [clj-opt*] [exec-opt] as the syntax -- -Spath is a clj-opt, -M is an exec-opt #2020-09-28 20:05:32 borkdude yep, makes sense, thanks for clearing that up #2020-09-28 20:05:59 seancorfield (as I just noted to @alexmiller it's not clear whether -A is meant to be a clj-opt or an exec-opt at this point, esp. given that at some future date it will stop running :main-opts) #2020-09-28 20:06:24 alexmiller It’s kind of in a gray area #2020-09-28 20:06:44 alexmiller But I think I would call it a clj-opt #2020-09-28 20:10:26 seancorfield At this point, I'm hoping that -A stops running :main-opts fairly soon and it can be clearly classified as a clj-opt and then -R and -C can be fully removed. #2020-09-28 20:33:00 borkdude @seancorfield I'm still getting the warning:
$ clojure -Sforce -Spath -A:glam
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
#2020-09-28 20:33:32 borkdude I don't want main, I want path #2020-09-28 20:33:43 borkdude I used to use -R -C for this #2020-09-28 20:34:19 borkdude can I rely on this being a correct invocation and the warning begone next version? #2020-09-28 20:34:29 seancorfield clojure -Sforce -Spath -M:glam should work. #2020-09-28 20:34:41 seancorfield -Spath implies -P I believe. #2020-09-28 20:34:43 borkdude it works, but I get the warning #2020-09-28 20:35:13 seancorfield (! 696)-> clojure -Sforce -Spath -M:runner
src:resources:/Users/sean/.gitlibs/libs/com.cognitect/test-runner/f7ef16dc3b8332b0d77bc0274578ad5270fbfedd/src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar:/Users/sean/.m2/repository/org/clojure/tools.namespace/0.2.11/tools.namespace-0.2.11.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
#2020-09-28 20:35:50 seancorfield Without -Spath that runs the tests (or attempts to -- it's missing the :test alias) #2020-09-28 20:36:01 borkdude aha, with -M it doesn't warn, but with -A it does... that's weird #2020-09-28 20:36:28 borkdude I think -A is the preferred way for creating paths right #2020-09-28 20:36:39 seancorfield (! 698)-> clojure -Sforce -M:test:runner
Running tests in #{"test"}
Testing tvaughan.example-test
FAIL in (a-test) (example_test.clj:7)
FIXME, I fail.
expected: (= 0 1)
actual: (not (= 0 1))
Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
(! 699)-> clojure -Sforce -Spath -M:test:runner
test:src:resources:/Users/sean/.gitlibs/libs/com.cognitect/test-runner/f7ef16dc3b8332b0d77bc0274578ad5270fbfedd/src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/test.check/1.0.0/test.check-1.0.0.jar:/Users/sean/.m2/repository/org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar:/Users/sean/.m2/repository/org/clojure/tools.namespace/0.2.11/tools.namespace-0.2.11.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
(! 700)->
🙂 #2020-09-28 20:37:14 borkdude anyway, I'll be using -A (and advocating it in my README) and expect the warning to be gone some time in the future #2020-09-28 20:40:01 borkdude clojure -Sforce -Spath -A:glam 2>/dev/null
#2020-09-29 15:53:21 borkdude fyi, the above is a cross-platform package manager for binaries written using Clojure, Windows support is coming soon (with the intent of not using Powershell) #2020-09-30 21:52:37 markbastian Is there a way to ignore the global ~/.m2/settings.xml when resolving repos? I have a situation in which my global file has a forced mirror that doesn't seem to be resolving snapshots correctly but if I just add the :mvn/repos to my deps.edn file and mv settings.xml settings.bak it works. #2020-09-30 21:53:18 markbastian But I don't want to have to do that every time I want to sync my snapshots to my internal repo. #2020-09-30 22:12:00 alexmiller That’s maybe the best option right now #2020-09-30 22:12:50 alexmiller You could also set that repo id to nil in your deps.edn and then add a new one? #2020-09-30 22:23:42 markbastian I’ll give that a shot. I’ve also got another workaround in which I run the clj command in a docker image with the source and ~/.m2/repository mounted in the container. Thanks! #2020-10-01 05:53:33 pmonks Apologies if this has been asked & answered before (I did search, but couldn’t find anything relevant), but in recent versions (specifically v1.10.1.697) the following occurs:
$ clj -e '(println "Hello, world")'
WARNING: When invoking clojure.main, use -M
Hello, world
$
I understand that this can be “fixed” with clj -M -e '(println "Hello, world")' , but is that intentional? #2020-10-01 08:27:38 practicalli It seems so. The -M flag is required when invoking clojure.main. #2020-10-01 16:26:37 pmonks That seems like an unfortunate UX decision… #2020-10-01 13:09:34 alexmiller yes, all invocations of clojure.main should now be routed through -M #2020-10-01 13:10:28 borkdude even when $ clj -- -e '(+ 1 2 3)' which is quite unambigious? #2020-10-01 13:11:10 alexmiller it's unambiguous now but we're trying to open up arg space for repl execution #2020-10-01 16:28:12 pmonks It seems to me that evaluation of a something provided on the command line should be considered at the same level of importance as running a REPL or invoking a main function, FWIW. #2020-10-01 17:16:17 alexmiller expression eval is done by invoking clojure.main. -M is how you (now) invoke clojure.main. #2020-10-01 17:21:41 vlaaad How are we going to invoke clojure.main in the future? :) #2020-10-01 17:21:55 alexmiller -M #2020-10-01 17:22:04 alexmiller where future = now #2020-10-01 17:34:41 vlaaad Ah, I misunderstood your previous message #2020-10-01 17:35:13 alexmiller yeah, I meant vs previous, not vs future #2020-10-01 17:34:43 alexmiller clj was originally built solely around clojure.main. we are in the process of making it bigger than that and this is part of segmenting off that part of the functionality. #2020-10-01 18:21:05 pmonks I understand the improvements that brings to the implementation, I’m more making a comment about the (negative, imho) UX that creates, especially for newcomers. #2020-10-01 18:21:47 pmonks i.e. why should a newcomer have to know that in order to evaluate an expression on the command line, they are actually invoking clojure.main? That seems like incidental complexity to me… #2020-10-01 18:22:10 pmonks To non-Java/Clojure folks, clj is clojure.main. #2020-10-01 18:22:27 alexmiller I don't think evaluating an expression on the command line is actually a thing most people do compared to running a repl #2020-10-01 18:25:45 pmonks Sure, but that doesn’t mean that CLI evaluation isn’t worth supporting in a clear and concise (and backwards compatible, though I know we’re in a period of time where that is being consciously and deliberately broken) manner. #2020-10-01 18:30:21 alexmiller is your complaint about the warning message mentioning clojure.main? #2020-10-01 18:30:35 pmonks No. #2020-10-01 18:31:20 alexmiller ok, then what is the complaint? #2020-10-01 18:31:36 pmonks My complaint is why does anyone need to know they’re invoking clojure.main in order to evaluate something on the command line. Shouldn’t this be a “first class” operation, regardless of how it’s implemented under the hood? #2020-10-01 18:32:03 pmonks After all that’s how most other languages handle this - Principle of Least Surprise etc. etc.
$ python -c 'print("Hello, world")'
Hello, world
$ perl -e 'print "Hello, world"'
Hello, world
etc. #2020-10-01 18:37:45 alexmiller ok, thanks for the feedback #2020-10-01 18:39:01 pmonks And to be clear, I don’t do this all that often, but it is a handy feature (e.g. in shell scripts). #2020-10-01 18:40:45 alexmiller the most common use of it that I'm aware of is to trigger deps loading without doing something else, which you can now do with -P #2020-10-01 22:19:13 pmonks Sure, but -P won’t evaluate an arbitrary expression. I like to (println a specific message to make build log searching easier. #2020-10-01 18:38:00 borkdude > I don't think evaluating an expression on the command line is actually a thing most people do compared to running a repl
I do this all the time on the command line #2020-10-01 18:38:56 borkdude -M -m -M foo.clj -M -e, don't know if I can get used it ;) #2020-10-01 18:38:57 alexmiller why vs running a repl? #2020-10-01 18:40:41 borkdude A lot of things I do in my open source projects is testing out command line applications which are usually one shot. Yes, I also use a REPL, but actually more often I execute one time invocations #2020-10-01 18:41:19 alexmiller so is that -e or -m ? #2020-10-01 18:41:49 borkdude let me just do a zsh history thing #2020-10-01 18:42:14 alexmiller (note new -P to prepare deps w/o executing, which is one prior use case for -e) #2020-10-01 18:52:17 borkdude ok, I wrote this horrible script now:
#!/usr/bin/env bash
history | bb -iO '(->> *input*
(map #(as-> % x (str/trim x) (str/split x #"\s+") (rest x) (take 2 x)))
(keep #(when (and (str/starts-with? (first %) "cl") (second %) (> (count (second %)) 1)) (subs (second %) 0 2)))
frequencies (sort-by second >) (take 10))'
["-A" 326]
["-S" 208]
["-e" 172]
["--" 79]
["-M" 76]
["/t" 30]
["-J" 22]
["-m" 18]
["-X" 12]
["-R" 8]
which indicates I use -e way more often on the command line #2020-10-01 18:52:36 borkdude but so far I haven't used -M a lot, usually -A #2020-10-01 18:53:38 borkdude because instead of -m there's usually an alias for it. But with -e there isn't. So when invoking main directly, -e is my primary use case. #2020-10-01 18:58:32 alexmiller thanks for the data #2020-10-01 19:03:12 borkdude I think clj /tmp/foo.clj is also one of my primary use cases, this is where ["/t" 30] probably comes from #2020-10-01 19:03:51 borkdude but because the filenames can change, it doesn't really show well here #2020-10-01 19:05:21 seancorfield I just checked my history, my dot-clojure file, and my deps.edn at work and I almost never use -e without also using -m it seems. Just as a contrary data point to the above. #2020-10-01 19:06:33 alexmiller thx #2020-10-01 19:07:15 borkdude $ alias cljm='clojure -M'
$ cljm -e "(+ 1 2 3)"
6
it works #2020-10-01 20:29:53 practicalli For my own brief experiences, I only every used -e once as a saw an example in the Clojure CLI docs. I am not aware of any use cases where I would want to evaluate clojure on the command line, especially if I can run the repl just as quickly. If I was going to script something with Clojure, then I'd try using Babashka, it looks great. I am fascinated as to what the scenarios could be for using the -e . If anyone has a link to examples, please let me know.
The only thing I find a little strange is when running an application from the command line, eg. clojure -M -m namespace.main , from a usability point of view the -M looks a little redundant.. After thinking about this for a few days, I realize I don't actually run the application that way. I'll either run in the REPL or create a jar/uberjar and run it that way. I though about using an alias called :project/run that ran the unpackaged project, which I could include in a project template (eg. for clj-new), but not sure how valuable that would be.
I find the -X flag more appealing (and more Clojurey) approach, as I've been writing command line -main functions to take hash-map as an argument for a while now. It seems an idiomatic approach to have all entry point functions in a project take a hash-map where possible. #2020-10-01 20:31:45 dpsutton in fish i have these handy shell functions
function clj-doc --description 'Get the docstring of a function'
clj -e "(require 'clojure.repl)(clojure.repl/doc $argv[1])"
end
one clj-doc and the other clj-source. easy to update to -M -e though #2020-10-01 20:31:45 borkdude Well, sometimes I use -e to require some namespace or a compile invocation for example #2020-10-01 20:32:47 borkdude No big deal, can be easily hidden in a script, Makefile or whatever, I just don't find -M -m aesthetically pleasing, which may not be an important argument. Whatever :) #2020-10-01 22:20:34 lread For what it is worth, I am updating rewrite-cljc to use the new cli syntax, and updating from -A -m to -M -m is not causing me any terrible discomfort. simple_smile #2020-10-01 22:28:52 markbastian Out of curiosity, what is the purpose of the sourceDirectory tag addition in the pom target? I don't see where it is used by other mvn tasks and the clojure-maven-plugin puts the same data into its own config (which I realize is unrelated to tools.deps, but it's the only mvn plugin I can think of that is directly related to building Clojure).
For example:
<build>
<plugins>
<plugin>
<groupId>com.theoryinpractise</groupId>
<artifactId>clojure-maven-plugin</artifactId>
<version>1.8.4</version>
<extensions>true</extensions>
<configuration>
<sourceDirectories>
<sourceDirectory>src</sourceDirectory>
</sourceDirectories>
</configuration>
</plugin>
</plugins>
<!-- How does maven use this? -->
<sourceDirectory>src</sourceDirectory>
</build> #2020-10-01 22:33:40 pmonks Pretty sure Maven uses that to know where to go to build Java source. IOW it’s probably not relevant for a deployed artifact. #2020-10-01 22:35:25 markbastian Ah, that makes sense - mixed language project. #2020-10-05 17:13:59 zetafish Hi all, I try to create an uberjar with depstar using the -X method. I.e. i updated my deps.edn to have an alias specifying a exec-fn and exec-arg. When running on my laptop clojure -X:uberjar a jar is generated just fine. However when I try to create the jar from a docker container it fails with
Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
-X:uberjar (No such file or directory)
The docker images I tried:
clojure:openjdk-11-tools-deps-1.10.1.469
clojure:openjdk-11-tools-deps-1.10.1.561
#2020-10-05 18:18:59 kirill.salykin it has interesting feature skinny jar #2020-10-05 19:54:15 zetafish I want a self contained jar that I can run with java -jar app.jar so skinny does not fit the bill. #2020-10-05 17:20:48 alexmiller you don't have the newest clj tools in your docker #2020-10-05 17:21:06 alexmiller you need clojure 1.10.1.697 #2020-10-05 19:52:00 zetafish Thank you Alex. I presume the 697 image will become available on docker hub soon. In the meantime I built the image locally and the -X:uberjar works fine then. #2020-10-05 20:01:35 alexmiller I don't have anything to do with the docker stuff, so no control over that for me #2020-10-05 17:57:39 seancorfield @zetafish I'm planning an overhaul of my project's readme files this week to make that clearer (that some of the usage examples require 1.10.1.697). I also need to update all of the clj-new project stuff to generate projects that can work both ways. #2020-10-05 18:19:07 dominicm Is clojure.libfile still present in 697? #2020-10-05 18:27:18 seancorfield I don't see it in the system properties when running code via 1.10.1.697 @dominicm -- there's clojure.basis now, which is the path to the combined EDN file. #2020-10-05 18:28:42 seancorfield I see it was still present in 590:
> /usr/local/Cellar/clojure\@1.10.1.590/1.10.1.590/bin/clojure
Clojure 1.10.1
user=> (System/getProperty "clojure.libfile")
"/Users/sean/.clojure/.cpcache/2731801087.libs"
user=>
#2020-10-05 18:28:57 dominicm Ah, that could explain the bug I'm seeing. I thought it was being retained for backwards compatibility. #2020-10-05 18:59:27 seancorfield @dominicm Lots of breakage between the previous stable and the current stable 🙂 but if you change (System/getProperty "clojure.libfile") to (some-> (System/getProperty "clojure.basis") (str/replace #"\.basis$" ".libs")) you can still get at the old file (although, as I recall, the basis file now contains the libs list as an element?) #2020-10-05 19:01:44 seancorfield Yeah, (some-> (System/getProperty "clojure.basis") (slurp) (edn/read-string) :libs) should be the same data. #2020-10-05 19:50:28 alexmiller yeah, I removed it in 697. we've never publicly doc'ed it and I was not aware of anyone using it (other than add-libs which won't work with latest anyways) so I didn't think there was any reason to retain it #2020-10-05 19:51:52 dominicm I was using it to find myself on the classpath and remove my paths #2020-10-05 20:00:51 alexmiller I'd say separating the tool from the basis is the better long term path for that #2020-10-05 20:01:11 alexmiller we have an ongoing conversation about some details of that right now #2020-10-05 20:21:00 dominicm If I could reliably identify my lib in the classpath, then that'd be fine 🙂. Happy to switch over to the newnew system when that comes out too. #2020-10-05 20:21:15 dominicm A github search indicates I'm probably the only user anyway #2020-10-05 20:21:30 dominicm We also use it in our internal tool based on the lib, but I can update that the same way #2020-10-05 21:20:48 seancorfield The problem with that (getting a file from the command line) is that the desired order for repos such as ours is:
System -> User (happy to exclude with -Srepro) -> repo-master (this is missing) -> Project -> -Sdeps
and because "last one wins" etc for some combinations of aliases, you can't always specify "repo-master` via some sort of -Sdeps-file option. #2020-10-05 21:24:14 seancorfield Although, if you could specify an arbitrary list of files to read in order via -Sdeps-file (and they didn't have to be called deps.edn) then we could use -Sdeps-file ../versions/master-deps.edn:project-deps.edn and not have an actual deps.edn file in each subproject in the repo. #2020-10-05 21:25:46 borkdude -Sdeps-path might be a more appropriate name and I'd be happy to support it in deps.clj (and change the name). It already does this now but for just one file #2020-10-05 21:27:10 borkdude -Sdeps-overrides could also work as a name to indicate that it doesn't read any other files than these #2020-10-05 21:29:55 borkdude but I'm relatively sure all of this can be done using scripting as well:
merge the configs yourself, provide it via -Sdeps and move the local deps.edn aside (hacky) #2020-10-05 21:31:22 borkdude also I'm relatively sure these things are in some row/column in an excel sheet somewhere on Alex's computer :) #2020-10-05 21:37:46 seancorfield Our deps files are pretty big -- too big to go on the command line I think as merged data -- and, as you say, that's a hacky approach. #2020-10-05 21:39:11 seancorfield (also, we're not about to switch from the official Clojure CLI to a third party tool at this point, esp when we would need multiple different binaries for different parts of our dev/test/staging pipeline, whereas we can have just one CLI install) #2020-10-05 21:41:20 seancorfield And, yeah, I've already brought this up with Alex (which was how I learned that it's similar to something the Datomic Ions users have asked for) so, yeah, I'll bet it's already in a spreadsheet. I don't know what I'd prefer in terms of a solution. As long as it addresses the deps ordering shown above and is officially supported, I'll be happy with whatever Cognitect/Nubank think is the best approach 🙂 #2020-10-05 21:43:38 borkdude @seancorfield small details: deps.clj also runs using clojure / JVM as a dependency but yeah I don't expect you to switch - I was just thinking out loud about this option. I'm blathering too much. #2020-10-05 22:01:02 seancorfield Ah, true, I'd forgotten there was a library version as well... #2020-10-05 22:01:52 borkdude I just pushed 0.0.10 - there was something with the clojars deploy token, missed that. But now it works:
/tmp $ clojure -Sdeps '{:deps {borkdude/deps.clj {:mvn/version "0.0.10"}}}' -M -m borkdude.deps -Sdeps-file "$HOME/git/clojurescript/deps.edn" -Spath
src/main/clojure:src/main/cljs:resources:/Users/borkdude/.m2/repository/com/cognitect/transit-clj/0.8.309/transit-clj-0.8.309.jar:/Users/borkdude/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20180805/closure-compiler-unshaded-v20180805.jar:... #2020-10-05 22:02:37 borkdude (so I'm in /tmp, but I'm using deps.edn from clojurescript in $HOME/git/clojurescript) #2020-10-05 23:47:16 alexmiller Just putting this here for now, but there is a new prerelease clj available - 1.10.1.708. Has a small fix to -X error handling and more importantly an update on how dep expansion works when multiple transitive versions of the same dep are in the tree to fix a case where a dep can be incorrectly omitted. #2020-10-07 15:42:55 alexmiller anyone tried this? #2020-10-07 16:45:21 seancorfield I'm about to update things here... #2020-10-07 18:52:55 seancorfield Building our uber JARs for our 14 processes seems to produce near-identical results between .697 and .708 -- there are variations from run-to-run based (I assume) on differing timestamps etc but nothing more than 150 bytes on JAR files that range from 30MB to 70MB, so that all looks reasonably sane. #2020-10-07 18:53:56 seancorfield (if dependencies were missing we'd have known about it by now on .697 and 150 bytes here and there isn't likely to indicate an entire dependency coming or going) #2020-10-07 18:59:12 alexmiller yeah, shouldn't be anything different other than the set of jars selected #2020-10-05 23:49:12 alexmiller The circumstances for the latter case are quite complicated so it's a little challenging for me to categorically say whether your deps are affected (they probably aren't), but if you were were interested in trying it, would be happy to have any feedback before I make it stable. #2020-10-07 11:58:58 timo for transitioning to clojure tools I am missing the lein install . How does one install a lib to local repository with clojure tools? #2020-10-07 11:59:53 dominicm @timok you don't really need to, you can reference it using :local/root #2020-10-07 12:00:17 timo 😆 easy as that. thanks @dominicm #2020-10-07 12:32:07 practicalli @timok as mentioned local/root is very useful. clojure -X:deps mvn-install is a built-in Clojure CLI alias to deploy a Jar of the project locally in the ~/.m2/repository directory #2020-10-07 12:37:49 dharrigan I find local/root very useful too. I pull down a 3rd party library, do some changes, and without having to deploy it anywhere, I can reference it in my own project by using the local/root structure to point to its sources. #2020-10-07 23:16:00 alexmiller yes, there have been changes in the clj arguments in 697 #2020-10-07 23:16:28 alexmiller those examples are (now) out of date, will fix #2020-10-07 23:16:41 alexmiller forgot they were there! #2020-10-08 15:44:44 alexmiller probably not, but I did think more about it - the time when it's read by clj is in tools.deps.alpha, which is the process that builds the classpath. at that point you don't have any deps on the classpath (b/c that's what you're making). so it is more complicated than it initially appeared. #2020-10-08 15:45:07 ghadi catch 22 #2020-10-08 15:45:30 rickmoynihan ahh so the data-readers aren’t even available yet? #2020-10-08 15:45:41 alexmiller no #2020-10-08 15:46:18 alexmiller tools.deps could do what's in the ticket, and that would open that up for library use of tools.deps. but it won't help you via clj. #2020-10-08 15:48:31 rickmoynihan not sure I understand. Would it not be possible to read it with a :default reader in the first pass, and then re-read it later when you have that info? #2020-10-08 15:57:09 alexmiller no? you would need to start a whole new jvm to do that #2020-10-08 15:52:20 borkdude @alexmiller With regards to env vars your reasoning was: the less tools have to know, the better it is. Doesn't that also go for reader tags? #2020-10-08 15:55:09 borkdude when opening up reader tags, one could implement one which reads env vars (or wait, it was resolving your home dir), which brings the same trade-offs maybe #2020-10-08 15:57:28 alexmiller yeah #2020-10-08 16:00:35 rickmoynihan Yeah I guess it makes sense not to do it then, thanks for considering it though 🙇 #2020-10-08 16:08:47 vlaaad what if reading tags is done with tagged-literal? #2020-10-08 16:09:22 vlaaad and then basis can be read with proper reader tags when there is a classpath.. #2020-10-08 16:10:19 vlaaad What’s the problem reader tags are supposed to solve though? #2020-10-08 16:14:44 rickmoynihan giving a data representation to otherwise un(print|read)able things. #2020-10-08 16:24:07 rickmoynihan So my use case is essentially:
I’m using integrant to define systems of components.
Those systems are typically wired together in a base.edn file on the classpath. That file contains a map of integrant components.
However it would be useful to override some parts (key/values) in the integrant system via the command line, and start the thing with clojure -X
Or in prodution allow a user to specify their own profiles as :exec-args data in the deps.edn file directly.
One of the things you may want to override though is an #ig/ref which basically states this component depends on this other component. My current example is a program that I want to communicate with two different deployments of the same system. One of which is authenticated via an auth0 provider, the other is identical but authenticated via a basic auth provider.
I was hoping tagged reader support in the cli tooling would help with this; but it seems I’ll need to implement something to do this. It’s not too complicated; but it’s something I could use in a lot of projects, so I was hoping I could just leverage the cli tools rather than reimplement it each time; or require a library function to do it. #2020-10-08 16:50:14 alexmiller You can read the deps.edn yourself and have any data you like in the aliases #2020-10-09 09:00:30 rickmoynihan I’d be happy to do that, but it would also be useful if the cli tools didn’t choke on it:
{:aliases {:reader {;;:exec-fn foo/bar
:exec-data {:tag #foo [some stuff]}}
}}
$ clj -X:reader
Error building classpath. Error reading edn. No reader function for tag foo (/Users/rick/tmp/deps.edn)
If they could set :default to clojure.core/tagged-literal this way would I think work for me.
Is that what you mean by this?
> A good problem would be to avoid any error like that while computing the classpath from the deps.
> That I would be happy to do #2020-10-09 09:01:14 rickmoynihan ahh also seen @U47G49KHQ appears to be suggesting this 👍 #2020-10-09 09:02:22 vlaaad yep, that’s what I suggested #2020-10-09 09:04:08 rickmoynihan Though I see now it wasn’t clear #2020-10-09 13:04:12 alexmiller Oh, I didn’t change it for -X param reading though, I’ll get that too before the next release #2020-10-09 13:15:15 vlaaad oh, that’s great!!! #2020-10-09 13:29:23 vlaaad #clj/requiring-resolve foo.bar/baz here I come #2020-10-09 13:30:52 rickmoynihan Ahh fab! Thanks @alexmiller 🙇 #2020-10-09 13:44:28 vlaaad Or bringing #= to edn with #clj/eval (some-code) #2020-10-08 16:51:04 alexmiller A good problem would be to avoid any error like that while computing the classpath from the deps. #2020-10-08 16:51:24 alexmiller That I would be happy to do #2020-10-08 16:51:46 vlaaad Hmm, agree, supplying non-data args via -X is complicated. My api functions in Reveal are supposed to only accept functions as arguments, but to make them available from clj-exec, I have to accept symbols as well and introduce polymorphism just to satisfy clj-exec: https://github.com/vlaaad/reveal/blob/master/src/vlaaad/reveal.clj#L11 #2020-10-08 16:52:37 vlaaad If deps supported tagged literals, I wouldn't have to pollute my api #2020-10-08 16:53:41 ghadi there is no straightforward way to do it. I wouldn't gate yourself on deps #2020-10-08 17:48:18 dharrigan I'm curious - what is the caching strategy used for looking up dependencies - I know that a particular dependency has been updated, it shows on clojars, but referencing it as 'RELEASE' in the deps.edn (my main deps.edn, i.e., $HOME/.clojure/deps.edn) I can see that it it's not being pulled down. #2020-10-08 17:48:51 hiredman use real versions #2020-10-08 17:49:20 borkdude @dharrigan it's a simple string hash on the user deps.edn, args and local deps.edn. Use -Sforce to update #2020-10-08 17:50:17 dharrigan Sure, I could do that - but I see for example in the popular seancorfield deps.edn, a lot of references to RELEASE as well. I mean it's not wrong to use RELEASE, it is supported. So surely, something must eventually attempt to find out if it is the latest release or not? #2020-10-08 18:36:58 seancorfield @dharrigan The only reason that some things in there use "RELEASE" is so that folks can "guarantee" a recent version of development tools when they first lift aliases from that file (or use the file whole). @U0NCTKEV8 is correct that this is "wrong" insofar as it breaks CLI caching and as @alexmiller says, this is not officially supported by the CLI/t.d.a so it's certainly not a good idea for anything other than quick access to dev tooling. I stopped using it in all the templates baked into clj-new a while back, although it is still used when looking up templates (on Clojars/Maven) to try to get the most recent stable release of a template -- and boot-new and lein new before it all use that same approach, with the same flaws: you can always use the -V / --version option to fetch a specific version of a template if you want/need. I am planning to make a pass over depstar and clj-new to make -X the favored way to invoke those now, including in the generated app template from clj-new, and at that point I'll also make a pass over my dot-clojure repo and start pinning versions, instead of "RELEASE", to ensure folks get a "recent-enough" version of things that actually support -X. Unfortunately, this sort of change is a big ball of yarn that causes a lot of stuff to need unraveling... #2020-10-08 17:50:39 hiredman it is wrong #2020-10-08 17:51:02 borkdude RELEASE is something that maven supports, but it's not encouraged by deps.edn, it just works because it works in maven. They will deprecate it. #2020-10-08 17:51:10 hiredman RELEASE is a mutable reference #2020-10-08 17:51:34 hiredman building caching on top of that is terrible, and clj relies on caching #2020-10-08 17:52:24 dharrigan Okay, fairy enough. I will use the version, not release 🙂 #2020-10-08 17:52:54 hiredman RELEASE is an easy vs. simple thing where people consistently choose what is easy #2020-10-08 17:53:03 dharrigan Indeed 🙂 We're all lazy 🙂 #2020-10-08 17:53:23 dharrigan $deity knows I didn't become a programmer to be active 🙂 #2020-10-08 17:53:26 alexmiller RELEASE is not officially supported by clj #2020-10-08 17:53:36 alexmiller use at your peril (it breaks the cache) #2020-10-08 17:54:26 dharrigan tbf, it (this library) was the only one in my deps.edn that has release for the coordinate #2020-10-08 17:54:33 alexmiller imo you should only ever use it in -Sdeps on the command line, not in a deps.edn file. I reserve the right to enforce that opinion at an arbitrary future time :) #2020-10-08 17:56:03 dharrigan Something else, does deps use the .m2/settings.xml at all? #2020-10-08 17:56:20 alexmiller yes, for some things (like server creds, mirrors, proxies) #2020-10-08 17:56:26 dharrigan right right, good. thanks 🙂 #2020-10-08 17:56:55 hiredman proxies for fetching maven deps, not for git deps? #2020-10-08 17:57:03 alexmiller yes, that is still a todo #2020-10-08 17:57:17 alexmiller there is a ticket for it, iirc #2020-10-08 18:01:27 borkdude I think it works in deps.clj because the proxy settings are also passed to the make-classpath invocation #2020-10-08 18:02:12 alexmiller that would make sense #2020-10-08 18:12:27 dpsutton i can get the version from clojure by running clojure --help but it prints with lots of other text obviously. is there a way to get just the version? #2020-10-08 18:13:05 dpsutton we have a script that invokes -m and wondering whether its better to omit the -M and get a warning or include it and break usage for people who are on older versions #2020-10-08 18:14:23 alexmiller clj -Sdescribe will return edn content that includes a :version key #2020-10-08 18:14:42 borkdude @dpsutton
$ clojure -Sdescribe | jet --query :version
"1.10.1.697"
#2020-10-08 18:15:03 alexmiller note that the warnings are on stderr so you can redirect stderr to null too if you want to not see those #2020-10-08 18:16:00 borkdude $ clojure -Sdescribe | clojure -M -e "(require '[clojure.edn :as edn]) (:version (edn/read *in*))"
"1.10.1.697"
#2020-10-08 18:16:37 borkdude $ clojure -Sdescribe | bb '(:version *input*)'
"1.10.1.697"
ok, I'm done now :) #2020-10-08 18:16:40 alexmiller heh #2020-10-08 18:17:04 alexmiller but I have all these cat skins lying about... #2020-10-08 18:18:54 borkdude k, one more:
$ clojure -Sdescribe | grep :version | cut -d' ' -f2
"1.10.1.697"
#2020-10-08 18:25:43 lread @dpsutton, I’ve moved to the new clojure command line syntax for rewrite-cljc. I have its scripts fail with an error message if clojure version < 1.10.1.697. My scripts are all in babashka so the check was easy. #2020-10-08 18:30:52 dpsutton thanks everyone #2020-10-08 18:38:27 dharrigan w00t. It will certainly help those coming fresh to Clojure and seeing your excellent deps.edn example. #2020-10-08 18:42:33 seancorfield Or the Practicalli example (nicer web site, more descriptive names, much better documentation). I'm considering linking to it from my dot-clojure repo and encouraging new folks to use the Practicalli stuff instead -- so I'm under less pressure to keep dot-clojure maintained and documented 🙂 #2020-10-08 21:29:48 practicalli @seancorfield I would be honoured to be linked from your repo. Thank you.
The library versions used in the Practicalli aliases are updated at least once a month using the :project/outdated alias and version changes added to the changlog.
Versions are updated sooner if I spot a new version or a library maintainer let's me know. #2020-10-09 00:29:26 seancorfield https://github.com/seancorfield/dot-clojure has been updated with caveats about it being my personal deps.edn file and to direct people to your repo if they want something well-documented and well-maintained! #2020-10-08 22:00:09 seancorfield I was planning to make -X the preferred way of using depstar in that release but I realized that -X:uberjar ends up putting the Clojure CLI's version of tools.deps.alpha into the JAR file -- because depstar assumes the "current classpath" is what you want in the JAR (and it excludes itself, assuming it is the only dependency you don't want in the JAR). -X:jar works just fine (because it excludes JAR files). I'll need to modify depstar to use t.d.a directly to build a classpath from deps.edn files (per the discussion the other day) but that gets complicated if you need a "user" deps.edn file and you're using CLJ_CONFIG with a monorepo -- as we are at work -- so that will have to wait for "depstar 2". Given the discussions around that and t.d.a itself recently in this channel, I'll probably defer that work until t.d.a has better support for that (since CLJ_CONFIG is part of the clojure script, not t.d.a). #2020-10-08 22:01:23 seancorfield BTW, CLJ_CONFIG is documented officially (I thought it wasn't):
User - cross-project configuration (typically tools)
Locations used in this order:
If $CLJ_CONFIG is set, then use $CLJ_CONFIG (explicit override)
https://clojure.org/reference/deps_and_cli#_deps_edn_sources #2020-10-08 22:13:32 hugod Is it possible to ignore a project’s deps.edn when invoking clojure cli? It would sometimes be useful when passing -Sdeps, e.g. in a tool that bumps a project’s version, or builds docs. #2020-10-08 22:14:15 seancorfield @hugod Yes, that's what :replace-deps is for. #2020-10-08 22:15:59 hugod thank you - I was looking for a command line switch #2020-10-08 22:16:24 seancorfield You can use it in -Sdeps but you'd need an alias I believe. #2020-10-09 19:00:16 seancorfield Sigh... seancorfield/depstar {:mvn/version "1.1.128"} and seancorfield/clj-new {:mvn/version "1.1.228"} are available now to fix the bug I introduced in depstar 1.1.126 in license file handling and for clj-new to use the fixed version of depstar; follow-up in #clj-new and/or #depstar (sorry for anyone who used yesterday's broken versions!) #2020-10-10 20:36:58 borkdude Made a small (proof of concept) tool to rewrite EDN files while preserving formatting, whitespace, comments, and reader ignores.
A few examples that might be handy for deps.edn: https://github.com/borkdude/rewrite-edn#examples #2020-10-11 13:59:44 Eamonn Sullivan Hi all. I have a real newbie question on deps.edn. I have the following to add cloverage to my project.
:cloverage
{:extra-paths ["test"]
:extra-deps {cloverage/cloverage {:mvn/version "RELEASE"}}
:main-opts ["-m" "cloverage.coverage"
"-p" "src"
"-s" "test"
"--fail-threshold" "90"]}
The cloverage docs say I should be able to provide a map of options somewhere, like using :fail-threshold instead of the args. How would I do that? #2020-10-11 14:16:42 alexmiller I don’t think this is a deps.edn question, not sure how cloverage works #2020-10-11 14:18:09 Eamonn Sullivan Ah, ok, sorry. I thought there was a key I could set like :options {:fail-threshold 90}. Told you it was a newbie question! #2020-10-13 15:50:08 tvaughan $ cat tests.edn
#kaocha/v1
{:plugins [:kaocha.plugin/cloverage
:noyoda.plugin/swap-actual-and-expected]
:cloverage/opts {:fail-threshold 100}} #2020-10-13 15:51:42 tvaughan @eamonn.sullivan ☝️ #2020-10-13 15:52:12 Eamonn Sullivan Ah, thank you. Perfect! #2020-10-11 14:19:28 Eamonn Sullivan You can set project default settings for Cloverage in your project. That's all they mention in the docs and I have no idea that means, either. I'll just use the command-line arguments. #2020-10-11 14:20:34 alexmiller Yeah, don’t know offhand #2020-10-11 14:27:19 lread @eamonn.sullivan, I don’t know have active it is, but there is a #cloverage channel. #2020-10-11 17:34:05 practicalli @eamonn.sullivan kaocha test runner can also use cloverage too. I have aliases for both, but I don't think I used options yet. Will give them a try and interested to know how you get on
https://practicalli.github.io/clojure-webapps/ #2020-10-13 11:09:42 orestis I’m seeing this in Github actions — when trying to do a -X — presumably some weirdness in Clojure version?
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
955
Could not locate clojure/run/exec__init.class, clojure/run/exec.clj or clojure/run/exec.cljc on classpath.
#2020-10-13 11:25:59 delaguardo how you setup clojure-cli for your action? #2020-10-13 11:26:26 orestis Via… your action 🙂 #2020-10-13 11:26:45 orestis clojure -Stree gives me this
org.clojure/clojure 1.10.1
1089
org.clojure/core.specs.alpha 0.2.44
1090
org.clojure/spec.alpha 0.2.176
1091
org.clojure/tools.deps.alpha 0.9.816
#2020-10-13 11:27:16 orestis And clojure --help gives me
Version: 1.10.1.708
#2020-10-13 11:27:46 delaguardo could you change the version of my action?)
to this DeLaGuardo/ #2020-10-13 11:28:52 delaguardo new jar (exec.jar) was introduced recently as a part of clojurecli installation
now it is fixed in master #2020-10-13 11:29:07 delaguardo sorry for inconvenience ( #2020-10-13 11:30:19 orestis oh 🙂 #2020-10-13 11:30:57 orestis No worries, I was just about to dive into the source code 🙂 #2020-10-13 15:14:52 flowthing Cool! Guess you could also do mvn versions:set -DnewVersion=1.0.1, too, though? #2020-10-13 15:15:39 borkdude Shhh #2020-10-13 15:15:58 borkdude Hehe. I didn't know that. ^ @UME7B1EBV #2020-10-13 15:16:40 borkdude I guess the script becomes more useful when you want to actually parse the version and bump it. #2020-10-13 15:19:55 flowthing Certainly. You might be able to do that with the Maven Release plugin, too, but I’m not sure. #2020-10-13 19:39:56 Søren Sjørup The maven version downloads everything on https://apache.org to do the job for me though 🙂 #2020-10-14 07:39:48 flowthing Sure, Maven downloading the entire internet before doing anything is a time-honored tradition. 🙂 It'll only do that the first time you run the command, though. #2020-10-13 15:22:03 alexmiller I think maybe I forgot to post it here, but I did a clj 1.10.1.716 prerelease this past weekend that includes a bump in many dependencies (maven resolver, aws api), and changes to make the edn reading more tolerant of unknown tagged literals in alias data #2020-10-13 15:25:11 seancorfield I'll update my dev/test setup today to try it out, and integrate it into our work repo. #2020-10-13 15:25:43 alexmiller thx #2020-10-13 19:20:34 seancorfield So far so good. Now it's on our QA infrastructure and handling deps/building JARs all seems to be working just fine. #2020-10-15 09:11:48 rickmoynihan ahh amazing I’ll make a note to give it a try at some point. Thanks 🙇 #2020-10-14 03:10:30 zane Can anyone think of a reason why someone might be getting USERAUTH fail when using clj to attempt to fetch a public repository via https? 😕 #2020-10-14 03:22:41 alexmiller If you could share a repro that would help #2020-10-14 03:24:18 zane Sure thing.
clj -Sdeps '{:deps {probcomp/metaprob {:git/url "" :sha "8dc9d09f747c1e29886bb9628a0110c6f6bc6f5a"}}}' #2020-10-14 03:24:25 zane Works for me. Fails for my colleague. #2020-10-14 03:24:35 zane I’ll get you the exact message. #2020-10-14 03:26:11 zane I’m confused as to why it appears to be attempting to fetch via git ssh. #2020-10-14 03:29:24 alexmiller well it's a git dep #2020-10-14 03:29:56 seancorfield FWIW, that command-line works fine for me on WSL2 (Ubuntu) on Windows 10 and also on macOS 10.12. #2020-10-14 03:31:01 seancorfield Can't test it on Powershell since I don't have the Clojure CLI installed on Windows natively any more. #2020-10-14 03:32:16 alexmiller I agree though that it does look like it's trying to use ssh though #2020-10-14 03:32:22 seancorfield @alexmiller Where could the jgit stuff be getting auth from that might cause a failure with a public git repo? #2020-10-14 03:32:44 alexmiller it's not that that it's not getting auth but that it's looking for it in the first place #2020-10-14 03:50:51 zane > well it’s a git dep
Sorry, what I meant was: I’m confused that it appears to be trying to fetch via SSH. #2020-10-14 03:51:02 zane Perhaps I’m reading this output wrong? #2020-10-14 03:52:41 alexmiller is there any chance that they have a ~/.gitconfig with a url.<base>.insteadOf set? #2020-10-14 03:53:43 alexmiller that allows you to direct git to use a different url instead of the original one and can be used to swap https for ssh urls #2020-10-14 03:53:54 seancorfield Or perhaps they have a deps.edn file in that directory that already specifies that library via {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "abccc2dfebccc2dfc3dec9"}, :content ("[email protected]")}? #2020-10-14 03:54:38 alexmiller the -Sdeps is merged last so that shouldn't affect anything #2020-10-14 03:54:44 seancorfield (they're running that command in their downloads folder so who knows what is in there) #2020-10-14 03:55:03 zane Yeah, it’s possible. #2020-10-14 03:55:16 zane I’ll ask. #2020-10-14 03:55:22 alexmiller should look for [url " and then an insteadOf line in /.gitconfig - something like that #2020-10-14 03:56:01 zane Got it. #2020-10-14 04:00:13 alexmiller [url "
#2020-10-14 04:00:19 alexmiller I was able to repro with that #2020-10-14 04:00:22 zane Seems like he’s signed off. I’ll check back in when I can get ahold of him again. #2020-10-14 04:00:33 zane I see! Hopefully that’s the culprit. #2020-10-14 04:00:42 zane Thanks for the help! #2020-10-14 04:01:00 alexmiller could even be more generic like: #2020-10-14 04:01:01 alexmiller [url "git://"]
insteadOf = https:// #2020-10-14 04:01:24 alexmiller can't say I've ever messed with that config before! #2020-10-14 04:02:30 zane Yeah, me either. #2020-10-14 04:03:21 zane The colleague having the issue is wanting to do some volunteer work on some of our private repositories, and I suspect he probably has a lot going on in his ~/.ssh/config from his job. #2020-10-14 04:05:27 zane From talking to Ghadi earlier I know how to set up a new machine for use with clj + private repositories, but I’m not clear yet on how to configure a machine that has a lot of extant ~/.ssh/config settings. #2020-10-14 04:06:53 alexmiller well, the answer is probably ... it depends #2020-10-14 04:07:08 zane For instance, I know that because of JSch clj doesn’t play well with IdentityFile declarations, and should instead be set up to read keys from the ssh-agent, but I don’t think I can ask this person to change their ~/.ssh/config in a way that might impact their work setup. #2020-10-14 04:08:09 zane > it depends
Yeah, I suppose there’s no way out but through. I’m going to have to understand what their existing ~/.ssh/config settings are doing in detail. #2020-10-14 18:36:46 zane To close this loop, he had this in his ~/.gitconfig:
[url "
#2020-10-14 18:37:02 zane Thanks, @alexmiller and @U04V70XH6! #2020-10-14 18:37:54 seancorfield Was he OK with removing that to get clj to work? #2020-10-14 18:38:32 seancorfield (I'm kind of curious as to why anyone would want all #2020-10-14 18:51:30 zane I’m curious too! #2020-10-14 18:51:33 zane He’s at his day job at the moment, so I won’t know more (why he had that in there, if he’s OK removing it) until this evening, probably. #2020-10-14 19:01:10 alexmiller 👍 #2020-10-15 03:32:54 zane Confirmed that that was the issue. Thanks again. :+1::skin-tone-2: #2020-10-14 19:18:22 kvlt What's the equivalent of lein install with deps.edn? #2020-10-14 19:29:12 seancorfield clojure -X:deps mvn-install #2020-10-14 19:29:44 seancorfield (you'll probably need to generate a pom.xml file first -- I can't remember) #2020-10-14 20:07:31 alexmiller no, you don't - you can either rely on the pom inside the jar or supply the g/a/v externally #2020-10-14 20:07:43 alexmiller or supply the pom #2020-10-14 19:29:56 seancorfield Oh, and that's assuming you're on 1.10.1.697 or later. #2020-10-14 19:33:28 dominicm @alexmiller how safe is it to rely on tdeps putting src/main/clojure on the classpath indefinitely? #2020-10-14 19:34:36 seancorfield @dominicm I didn't think it did? :paths ["src"] is in the system deps #2020-10-14 19:36:01 dominicm Oops, via pom.xml* #2020-10-14 20:06:58 alexmiller I'm confused by what you mean #2020-10-14 20:21:13 dominicm @alexmiller there's always src/main/clojure added when doing a local/root to a pom.xml right now. #2020-10-14 21:46:20 alexmiller So local dep with pom.xml #2020-10-14 21:46:36 alexmiller Not planning to change that #2020-10-15 01:07:14 Jacob O'Bryant The new git-resolve-tags command appears to have a bug. Someone reported this to me today and I've been able to reproduce as well:
$ cat deps.edn
{:deps
{github-jacobobryant/biff
{:git/url ""
:tag "HEAD"}}}
$ clj -X:deps git-resolve-tags
Error building classpath. Library github-jacobobryant/biff has :tag but no :sha.
Add :sha or run `clj -Sresolve-tags` to update deps.edn.
$ clj -Sresolve-tags
Option changed, use: clj -X:deps git-resolve-tags
$ clj --help | head -n 1
Version: 1.10.1.697 #2020-10-20 21:24:45 alexmiller fix for this will be in the next prerelease version, coming shortly #2020-10-20 21:37:17 alexmiller available in 1.10.1.723 #2020-10-15 03:03:59 alexmiller heh, thanks for the report #2020-10-19 00:27:30 athomasoriginal Is there an equivalent of :local/root in the alternative tools e.g. lein or boot?
The reason for the question is because this makes debugging 3rd party library (say reagent) wildly easy. Just clone the lib, add it as a :local/root and then start editing the library as if it was part of your project. Seems like reason enough for Clojure Tools to be used over alternatives. #2020-10-19 00:36:09 seancorfield There's also a plugin for Leiningen that lets you specify your dependencies via a deps.edn which, presumably, would also serve you for this purposes I think... #2020-10-19 00:39:37 seancorfield (`checkouts` would require that the checked-out library be a Leiningen project as well -- Rick's plugin should work for non-lein projects, but for transitive dependencies you'd need it to have pom.xml or deps.edn in it I think) #2020-10-19 21:48:57 grzm Was there a change recently (past month or so) in clojure.tools.deps that changed the order of the classpath? In clojure 1.10.1.507 I’m seeing clojure -Spath puts deps.edn :paths first (e.g., with {:paths ["src/main/clojure"] ,,,}, src/main/clojure is the first item in the classpath), and in clojure 1.10.1.645 I’m seeing :paths listed last. #2020-10-19 21:50:38 dpsutton > 0.9.799 on Sep 16, 2020- Fix classpath ordering for paths and libs
#2020-10-19 21:51:19 grzm Cheers! Is there a quick way to correlate tools.deps versions other than looking at the change log? #2020-10-19 21:52:09 dpsutton i'm not sure. i just remembered that change being discussed vaguely. sorry #2020-10-19 21:52:20 dpsutton correlate versions to what? #2020-10-19 21:52:22 grzm No worries, you already did more than I could do on my own 🙂 #2020-10-19 21:53:20 grzm The clojure-tools release and the clojure.tools.deps releases. Looking at the clojure -h on an installation, I see the former. I’d like to get at the latter more easily. #2020-10-19 22:27:46 alexmiller That’s kind of a consolidated release page for clj and tools.deps (versions of that included for reference) only for stable versions of clj #2020-10-20 18:49:45 grzm Yeah, found that. I was hoping to have something similar to the output of clojure -h on the command line, to aid in debugging rather than referring to a webpage. That works, though. #2020-10-20 18:54:44 alexmiller you can tell by doing clj -A:deps -Stree | grep tools.deps.alpha #2020-10-20 19:54:38 grzm Cheers! #2020-10-20 19:55:20 grzm writes that down #2020-10-20 00:22:18 cch1 I’m getting up to speed with the deploy process using tools from the greater tools.deps ecosystem and I ran into a strange issue today. Essentially, I was finding that libs installed locally did not bring their transitive dependencies along with them. I used clojure.tools.cli.api/mvn-install to do the install. #2020-10-20 00:24:05 cch1 A bit of investigation shows that mvn-install does not copy the pom file to the local repo -interestingly @seancorfield has observed that slipset/deps-deploy does install the pom file. So is that a bug or a limitation/design decision in c.t.d. mvn-install? #2020-10-20 00:24:09 cch1 @alexmiller ^ #2020-10-20 00:32:29 seancorfield @alexmiller I confirmed with @cch1 that running clojure -X:deps mvn-install omits the .pom file and when trying to use that dependency in a clean folder, t.d.a. reaches out to Clojars to look for the "missing" .pom file. When I re-ran the same scenario but using slipset/deps-deploy for the local install, it did copy pom.xml to <library>.pom in ~/.m2 and when trying to use that in a clean folder, t.d.a. did not reach out to Clojars to look for the .pom file. Definitely feels like a bug in t.d.a's mvn-install to me. #2020-10-20 00:43:32 alexmiller can you back up to the full command you're running? #2020-10-20 00:44:07 alexmiller are you supplying a pom? using the one in a jar file? supplying additional attributes? #2020-10-20 00:45:37 seancorfield There is a pom.xml in the same directory. I ran just clojure -X:deps mvn-install :jar '"the-lib.jar"' -- do we need extra args to also copy the pom.xml file? It reads the pom.xml file to get the group/artifact/version data... #2020-10-20 00:46:42 seancorfield The same pom.xml file is inside the JAR as well (so that's the one it reads, based on the reference docs). #2020-10-20 00:48:24 seancorfield Ah, if I say :pom '"pom.xml"' as well, then it does copy that to <library>.pom -- is that an intentional difference @alexmiller? #2020-10-20 00:53:29 seancorfield Here's a full repro session:
(! 692)-> rm -rf ~/.m2/repository/seancorfield/
(! 693)-> jar tvf next-jdbc.jar |fgrep pom
110 Mon Oct 19 11:24:06 PDT 2020 META-INF/maven/seancorfield/next.jdbc/pom.properties
1996 Mon Oct 19 11:24:06 PDT 2020 META-INF/maven/seancorfield/next.jdbc/pom.xml
(! 694)-> clojure -Sforce -X:deps mvn-install :jar '"next-jdbc.jar"'
Installing next-jdbc.jar
Installed to /Users/sean/.m2/repository/seancorfield/next.jdbc/1.1.610
(! 695)-> ls -lR ~/.m2/repository/seancorfield/
total 0
drwxr-xr-x 4 sean staff 136 Oct 19 17:51 next.jdbc
/Users/sean/.m2/repository/seancorfield//next.jdbc:
total 8
drwxr-xr-x 4 sean staff 136 Oct 19 17:51 1.1.610
-rw-r--r-- 1 sean staff 305 Oct 19 17:51 maven-metadata-local.xml
/Users/sean/.m2/repository/seancorfield//next.jdbc/1.1.610:
total 96
-rw-r--r-- 1 sean staff 164 Oct 19 17:51 _remote.repositories
-rw-r--r-- 1 sean staff 44300 Oct 19 11:24 next.jdbc-1.1.610.jar
(! 696)-> rm -rf ~/.m2/repository/seancorfield/
(! 697)-> clojure -Sforce -X:deps mvn-install :jar '"next-jdbc.jar"' :pom '"pom.xml"'
Installing next-jdbc.jar and pom.xml
Installed to /Users/sean/.m2/repository/seancorfield/next.jdbc/1.1.610
(! 698)-> ls -lR ~/.m2/repository/seancorfield/
total 0
drwxr-xr-x 4 sean staff 136 Oct 19 17:52 next.jdbc
/Users/sean/.m2/repository/seancorfield//next.jdbc:
total 8
drwxr-xr-x 5 sean staff 170 Oct 19 17:52 1.1.610
-rw-r--r-- 1 sean staff 305 Oct 19 17:52 maven-metadata-local.xml
/Users/sean/.m2/repository/seancorfield//next.jdbc/1.1.610:
total 104
-rw-r--r-- 1 sean staff 188 Oct 19 17:52 _remote.repositories
-rw-r--r-- 1 sean staff 44300 Oct 19 11:24 next.jdbc-1.1.610.jar
-rw-r--r-- 1 sean staff 1996 Oct 19 11:16 next.jdbc-1.1.610.pom
(! 699)-> ls -l pom.xml
-rw-r--r-- 1 sean staff 1996 Oct 19 11:16 pom.xml
#2020-10-20 00:54:50 seancorfield I see Installing next-jdbc.jar in the first case and Installing next-jdbc.jar and pom.xml in the second case so I suppose it is telling you what it is doing... #2020-10-20 00:56:19 seancorfield Without the .pom file in the repo, attempts to use the dependency will still reach out to Clojars/Maven for that .pom file and if it isn't found, it doesn't look like the pom.xml inside the JAR is sufficient for t.d.a to pick up the dependencies. #2020-10-20 01:01:04 alexmiller reaching out to maven is surprising though, so I'll look at that tomorrow #2020-10-20 01:03:03 seancorfield Downloading: seancorfield/next.jdbc/1.1.610/next.jdbc-1.1.610.pom from clojars
^ That's what I get trying to use the installed dependency without the .pom file in the repo. If I install the pom.xml file as well (or use slipset/deps-deploy install) so that I get the .pom file in the repo, that line does not appear when trying to use the dependency. #2020-10-20 01:03:27 seancorfield Feel free to DM me tomorrow if you need me to run more test cases. #2020-10-20 01:03:48 seancorfield (and there was no attempt to download the .jar file, only the .pom file) #2020-10-20 01:04:52 alexmiller sorry, maven looking for the pom file is normal behavior, I thought you were saying it was reaching out for the pom file when installing #2020-10-20 01:05:45 seancorfield Right, but that will fail if there is only a local install -- and the problem Chris was seeing was that if no .pom file existed, transitive dependencies were not being found. #2020-10-20 01:06:15 alexmiller well that's to be expected #2020-10-20 01:06:24 alexmiller the question is what you supply on install #2020-10-20 01:06:43 alexmiller you have to either supply a pom, or use the pom in the jar, or is this the case where you did neither? #2020-10-20 01:07:03 seancorfield There is a valid pom.xml inside the JAR -- I showed that above. #2020-10-20 01:07:27 alexmiller sorry, too much reading, and I'm a drink into the night :) #2020-10-20 01:07:43 seancorfield The problem is that the .pom file is not created in the local repo on the install. Unless you explicitly specify :pom '"pom.xml"' #2020-10-20 01:08:11 alexmiller ok, that's surprising to me, should be using the one in the jar in that case #2020-10-20 01:08:12 seancorfield OK, we can follow-up tomorrow. Feel free to DM me about it. #2020-10-20 01:08:44 seancorfield See my console session above. #2020-10-20 01:23:10 alexmiller yeah, that code is not there. it is using the pom in the jar to pull G/A/V but it's not actually installing it as a distinct artifact. not sure why I thought that would happen automatically, maybe tricked myself with some caching or something #2020-10-20 01:23:53 seancorfield So, definitely a bug? Want me to create a JIRA? #2020-10-20 01:24:02 alexmiller definitely a bug, sure on jira #2020-10-20 01:24:06 alexmiller should be easy to fix #2020-10-20 01:42:20 seancorfield seancorfield/depstar {:mvn/version "1.1.132"} -- calls shutdown-agents at the end to account for "badly-behaved" code being AOT'd that has side-effecting top-level forms that start threads (e.g., Neanderthal) -- follow-up in #depstar #2020-10-20 01:52:08 cch1 Sorry I’m late catching up here. I don’t have much more to offer than what Sean noted above. Let me know if I can help test. #2020-10-20 21:36:02 alexmiller Clojure tools 1.10.1.723 (a prerelease) is now available:
• Fix clj -X:deps tree adding tools.deps.alpha to tree
• Fix clj -X:deps mvn-pom adding tools.deps.alpha to pom deps
• Fix clj -X:deps git-resolve-tags not working
• Fix clj -X:deps mvn-install on jar to also install embedded pom
#2020-10-20 21:47:15 alexmiller the first 3 were actually all a bug in the default :deps alias using :extra-deps rather than :replace-deps (duh) #2020-10-21 14:03:29 alexmiller if anyone tries that prerelease and has good/bad feedback, please let me know, I will probably release it as stable by tomorrow #2020-10-21 16:35:14 seancorfield I updated our repo at work to use it immediately but I haven't exercised it much yet (and in particular I haven't exercised the options you fixed yet!). I'm just about to update our "ancient" script which uses -Stree to use -X:deps tree instead and see if it still works correctly... #2020-10-21 16:36:12 seancorfield Oh, I misspoke. Apparently I made that change a while back so I have already tested that it didn't break anything... #2020-10-21 16:37:19 seancorfield ...and I can confirm that all the maven-resolver-* deps disappeared, as expected. So, yup, that worked. #2020-10-21 16:44:44 seancorfield OK, I just ran clojure -Spom across a dozen projects and then clojure -X:deps mvn-pom and confirmed there's no differences between those two now. #2020-10-21 16:49:43 alexmiller there is still a difference in that your command line classpath-modifying arguments won't change the generated pom with the latter #2020-10-21 16:50:58 seancorfield Right, I don't rely on that in any of the projects where I create pom.xml files at the moment. #2020-10-21 16:51:16 seancorfield But, yeah, we couldn't use this in our monorepo. #2020-10-21 16:52:22 alexmiller still a TBD but I think we will probably have some semi-standard set of args that you can use to do that on mvn-pom if desired (or in tree). not sure yet whether that is args that mirror clj switches or pass through of some kind that gets parsed etc. #2020-10-21 16:56:46 seancorfield That did cause me to notice that
CLJ_CONFIG=../versions clojure -A:defaults -Spom
and
CLJ_CONFIG=../versions clojure -R:defaults -Spom
behave differently. The latter works, in that it merges the :defaults alias (with all our :override-deps) into the lib map which -Spom sees but the former fails to do that. #2020-10-21 16:58:01 seancorfield With -A:defaults I get this error
Error generating pom manifest: No coordinate type found for library compojure/compojure in coordinate {}
because that's how we specify common deps that have :override-deps in the :defaults alias in our "master" deps.edn file. #2020-10-21 18:21:01 alexmiller I can see why that is - the gen script piggybacks the normal cp script and one of the arg names changed in the latter so it's getting missed #2020-10-21 18:21:34 alexmiller at the time that got changed, the gen script had been removed so didn't show up in the refectoring but was later re-added #2020-10-21 18:38:34 alexmiller @seancorfield fixed in 1.10.1.727 #2020-10-21 18:39:15 seancorfield I will update to that shortly 🙂 #2020-10-21 19:21:49 seancorfield Updated. Confirmed fixed. Thanks @alexmiller #2020-10-22 10:05:12 practicalli Should the -M flag for Clojure CLI tools have an issue when also supplying -Sdeps options? (the -A flag works without issue for the same command)
/usr/local/bin/clojure -M:env/test -Sdeps '{:deps {}}'
This gives an error, whether or not the :deps contains any dependencies.
Execution error (FileNotFoundException) at java.io.FileInputStream/open0 (FileInputStream.java:-2).
-Sdeps (No such file or directory)
I first experienced this issue when running Clojure CLI tools with Cider (Emacs) and using the -M flag, specifically with cider-jack-in which injects the relevant dependencies for cider-nrepl using `-Sdeps argument. I can disable the actual dependencies, but not yet
Testing the same command on the terminal, gives the same error.
Using the command without the -M:env/test alias, only the -deps arguments works
Using the -A flag works with S-deps argument
Starting a repl with that command, minus the -Sdeps argument works via the command line and I can connect
I have tried this with several versions of Clojure CLI tools installed, from 1.10.1.697 onwards, including the latest pre-release. All seem to have the same issue. #2020-10-22 10:14:59 borkdude @jr0cket The -S... options should go before -M #2020-10-22 10:16:36 borkdude clj-opts before exec-opts (although they are mentioned in the other order in the help text, that order is documented as Run main clojure [clj-opt*] -M[:aliases] [init-opt*] [main-opt] [arg*] ) #2020-10-22 10:19:13 borkdude (I've made the same mistake before) #2020-10-22 10:20:57 practicalli Yes, putting the -M after -Sdeps does seem to work on the terminal.
That ordering is not the case with -A flag and so sounds like a change is required for Cider to work correctly with jack-in. The ordering can be changed by editing the the command line each time, but using .dir-locals.el config to set the alias, the -M and -A flags are put before the -Sdeps option
I assume -A will also work after -Sdeps option, to support those who havent upgraded to Clojure CLI tools #2020-10-22 10:21:43 borkdude the same rules apply for -A #2020-10-22 10:22:11 borkdude Start a REPL clj [clj-opt*] [-A:aliases] [init-opt*]
#2020-10-22 10:23:11 practicalli Except -A doesnt uphold that rule in practice... #2020-10-22 10:23:35 borkdude It's a case of Hyrum's law, but I guess the tools doing it wrong should be changed. #2020-10-22 10:27:17 practicalli Confirmed that editing the cider command line and putting -M flag in the correct place does work :)
/usr/local/bin/clojure -Sdeps '{:deps {nrepl {:mvn/version "0.8.2"} refactor-nrepl {:mvn/version "2.5.0"} cider/cider-nrepl {:mvn/version "0.25.3"}}}' -M:env/test -m nrepl.cmdline --middleware '["refactor-nrepl.middleware/wrap-refactor", "cider.nrepl/cider-middleware"]'
I'll raise an issue with Cider to see if they can fix this. Thanks. #2020-10-22 11:47:11 dominicm That's annoying. It looks like we'll need to write a parser for the arguments or something and split on it? Not too terrible, but it is if you want to be backwards compatible. #2020-10-22 11:51:35 dominicm Hmm, I guess I'll have to check if my scripts use prefix or suffix form. That pattern is used in other tools too. #2020-10-22 12:09:28 borkdude or just don't use cider-jack-in. I never use it. #2020-10-22 13:22:32 dominicm I do. It's a useful tool, especially for beginners. Having less pieces to understand is very useful. #2020-10-22 13:24:17 borkdude Having less magic to understand is also useful ;) #2020-10-22 13:26:25 dpsutton I also really like that CIDER can start up almost any project for you automatically without thought. It displays its startup command in the repl so you know there actually isn’t any magic. It’s just an extra dep and a main so you can run from the clj easily #2020-10-22 13:54:24 dominicm There's no magic, just trying to build on top of simple tools underneath. It's more like an abstraction. #2020-10-22 13:54:57 borkdude It feels like too much magic to me. Also I don't want the REPL to be a subprocess of my editor.. #2020-10-22 17:33:52 dominicm implementation detail, subproc part could change :) #2020-10-22 13:55:56 dpsutton the subprocess part is fair 🙂 If there's a problem in your lisp interpreter from 1970 your repl goes down ... #2020-10-22 14:12:38 alexmiller Clojure tools (clj) https://clojure.org/releases/tools#v1.10.1.727 is now available:
• Fix clj -X:deps tree adding tools.deps.alpha to tree
• Fix clj -X:deps mvn-pom adding tools.deps.alpha to pom deps
• Fix clj -X:deps git-resolve-tags not working
• https://clojure.atlassian.net/browse/TDEPS-152 - Fix clj -X:deps mvn-install on jar to also install embedded pom
• Fix clj -Spom not respecting dep modifications from -A (regression)
#2020-10-22 17:32:42 rschmukler Is there any way to install dependencies from a given alias without running an associated :main-opts in the alias? Essentially I want a -e "" that ignores the main-opts #2020-10-22 17:33:12 rschmukler (specifically from the CLI, I realize its very doable via the clj package) #2020-10-22 17:42:19 rschmukler Right now my best shot is using babashka to construct a new deps.edn and invoking clj -Sdeps
clj -Sdeps "`cat deps.edn | bb -I -e "{:deps (merge (:deps (first *input*)) (get-in (first *input*) [:aliases :native-image :extra-deps]))}"`" -e ""
But this feels like quite the hack #2020-10-22 17:48:57 borkdude @rschmukler (I don't think you need -I here, since you're reading only one EDN value) #2020-10-22 17:49:54 borkdude cat deps.edn | bb -e "{:deps (merge (:deps *input*) (get-in *input* [:aliases :native-image :extra-deps]))}" #2020-10-22 17:50:16 rschmukler Right on! I misunderstood the CLI. I thought -I coerced to EDN from string literal #2020-10-22 17:50:39 borkdude -I is for reading multiple EDN values from stdin #2020-10-22 17:52:06 seancorfield So clojure -P -M:my:aliases:here assuming you're using a recent release of the CLI. #2020-10-22 17:52:40 rschmukler @seancorfield perfection! Sorry that I missed that. Might be worth adding that to the clj --help docs too #2020-10-22 17:53:06 seancorfield It's in that help
exec-opts:
-A:aliases Use aliases to modify classpath
-X[:aliases] Use aliases to modify classpath or supply exec fn/args
-M[:aliases] Use aliases to modify classpath or supply main opts
-P Prepare deps - download libs, cache classpath, but don't exec
#2020-10-22 17:53:36 rschmukler Then clearly my clj must be outdated lol! #2020-10-22 17:53:47 rschmukler Right'o - thank you for the help! #2020-10-22 17:53:57 seancorfield (! 652)-> clojure -Sdescribe
{:version "1.10.1.727"
:config-files ["/usr/local/Cellar/ #2020-10-22 17:54:15 rschmukler :version "1.10.1.645
#2020-10-22 17:54:21 rschmukler There we are #2020-10-22 17:56:03 seancorfield Yeah, 645 was one of the prerelease builds while a lot was changing in the CLI. #2020-10-22 17:59:10 rschmukler Now I've just gotta figure out why Manjaro hasn't kept up #2020-10-22 18:08:04 rschmukler Updated my clojure and sure enough, it's all there for me. Thanks again @seancorfield #2020-10-22 18:18:35 practicalli Clojure CLI 1.10.1.697 is the minimum version for the -P and new mode for the -M flags. 1.10.1.727 is the latest (officially release today) #2020-10-22 18:19:54 seancorfield I'll be glad when -A changes to no longer run :main-opts 🙂 #2020-10-22 18:26:18 seancorfield @U064X3EF3 This is still the plan, at some point, right? #2020-10-22 18:31:53 alexmiller yes #2020-10-22 18:32:10 alexmiller I expect that to be ~months #2020-10-22 22:11:29 seancorfield @jr0cket I see
Run the project clojure -X:project/run -m domain.main-namespace
but there's no :project/run alias in your deps.edn so I'm not sure what this is meant to do? (and it seems weird to have -X and then have -m domain.main-namespace as well?) #2020-10-22 22:12:36 seancorfield Also, you have
Package library clojure -X:project/jars
That should be -X:project/jar singular, yes? #2020-10-23 09:35:04 practicalli Thanks for the review, those issues are now fixed, along with further explanation.
Using a :project/run alias seems to make most sense as a project level deps.edn, so the specific namespace and function names are in the config. I'm adding this to the templates I'm creating with clj-new. #2020-10-22 22:03:41 drewverlee When I run clj i get an exception about building the class path. I tried deleting my cpcache. If you can think of something else i can try please chime in.
gist file is my deps.edn. the comment contains the command run and output (error): https://gist.github.com/drewverlee/893a150c780a86fbf73cd789b48d9b40
Thanks in advance. ill post in this thread if i figure it out. #2020-10-22 22:05:27 dpsutton #:com.datomic{ion-dev #:mvn{:version "0.9.276"}} the ion-dev does not begin with a colon but in #:mvn{:version it does. can you try fixing that? #2020-10-22 22:10:06 dpsutton scratch that. namespaced symbols look strange to me in that fashion 🙂 #2020-10-22 22:12:25 drewverlee It is strange, I'll clean it up. But I'm guessing that's not it. That's from copy pasting #2020-10-22 22:12:50 alexmiller I don't think that's it - this look like maybe a bad version range in some transitive dep? #2020-10-22 22:13:16 alexmiller although I know all those deps so not sure what you'd find there :) #2020-10-22 22:14:44 seancorfield {:deps #:com.datomic{ion-dev #:mvn{:version "0.9.276"}}
under an alias, that should be :extra-deps, not :deps (or :override-deps) #2020-10-22 22:15:02 alexmiller typo: mvn/verison #2020-10-22 22:15:29 seancorfield Ah yes, on com.datomic/ion {:mvn/verison "0.9.48"} #2020-10-22 22:15:29 alexmiller newest version of clj will give you a better error message there #2020-10-23 00:08:38 drewverlee i see the typo now. ugh. I should turn on a spell check before i post something to catch things like this. #2020-10-23 00:42:49 alexmiller On latest clj, you'll get: Error building classpath. No :mvn/version specified for com.datomic/ion #2020-10-23 07:08:51 dominicm Now that add-lib isn't functional, if I wanted to load dependencies for a particular function call with exec-opts (without the user having to know which dependencies to add to deps.edn for that call) how would I do that?
I want to provide some beginner tools as part of something, but I don't want to bloat the primary use case.
If I had bash-style exec, I'd probably just restart the jvm that way.
I want the set of functionality to be an open-set though, so it might expand in the future, and the user shouldn't need to fiddle around to get that working. I'm certainly aiming for "easy" here. #2020-10-23 07:24:05 dominicm Just looked at what ion does, that has ion-dev which you install in your user deps.edn, that strikes me as an odd choice. #2020-10-23 10:14:31 dominicm I suppose this could be considered the ability to import deps.edn aliases into your project somehow :) #2020-10-23 11:17:23 alexmiller Putting ion-dev in user deps.edn is recommended - it’s a general tool for deploying ions #2020-10-23 11:20:27 alexmiller We will be updating add-lib to work again btw with some additional apis and ultimately in master #2020-10-23 11:21:05 alexmiller But not sure I have a good answer for this use case right now #2020-10-23 11:46:22 dominicm np. Maybe I'll start with bloat, then swap to add-lib later. #2020-10-23 16:41:34 seancorfield I built a version locally, using Maven, based on a t.d.a add-lib branch a while back I think. Whatever was the equivalent of 0.8.667 it seems. #2020-10-23 16:45:46 seancorfield I built it on April 6th, so it's probably the head of the add-lib branch, which was 0.8.667 plus the calc-basis stuff, looking at the commit logs. @dominicm I guess keeping your own version around for this isn't going to work for your use case? #2020-10-23 16:46:46 dominicm @seancorfield it would yeah, I had assumed that stuff was completely broken by the internal changes. I'll give it a go, thanks! #2020-10-23 16:50:52 seancorfield -rw-r--r-- 1 sean admin 58110 Apr 6 2020 tools.deps.alpha-0.8.667-add-lib.jar
In case you want a sanity check on the size of the JAR after you build it. Then I just use :local/root to depend on the JAR. #2020-10-23 16:52:38 dominicm I'll probably have to deploy it somewhere, but that's fine :) #2020-10-23 19:03:21 alexmiller I just pushed an add-lib3 branch that's up to date with master #2020-10-23 19:04:49 alexmiller note that add-lib is now add-libs and there are some other additional apis (not final) in there too - find-revs and find-lib #2020-10-23 19:08:26 alexmiller also, at some point in the past I made it degrade gracefully if it doesn't have the compiled s3 class stuff so it's a little easier to hack onto a classpath like it was originally #2020-10-23 19:10:26 dominicm Great, I guess I know what I'm doing tomorrow then! #2020-10-23 19:11:31 dominicm My goal is that beginners can do (require 'beginner) or something, and it'll go ahead and fetch tools like spyscope and dot-clojure2 for them, and load them up.
Perfect compromise between beginners and advanced users (who will set that stuff up for themselves eventually) #2020-10-23 21:29:35 seancorfield @alexmiller Thanks for the add-lib3 branch -- very nice that it works from source again, albeit without S3 support! Just updated my .clojure file for that. #2020-10-23 08:30:03 Eamonn Sullivan Good morning, does anyone use CircleCI with tools-deps and have an example I can see? I'm just exploring a few alternatives to our Jenkins. I use Github actions on my own stuff (e.g., https://github.com/eamonnsullivan/github-pr-lib I create for scripts and deploy to clojars) and I'm also going to take a look at AWS CodeBuild/Pipeline, but examples to study are mostly lien-based. #2020-10-23 08:33:41 borkdude @eamonn.sullivan Almost all of my projects do #2020-10-23 08:34:06 Eamonn Sullivan Ah, perfect. Thank you! #2020-10-23 09:16:01 practicalli @eamonn.sullivan There are images you can choose from that already have Clojure CLI tools included.
https://practicalli.github.io/clojure/testing/integration-testing/circle-ci/
If you want the very latest version of Clojure CLI tools, then use Borkdude's approach (which I am going to add to that page I mentioned right now, as I want to use 1.10.1.727) #2020-10-23 19:59:42 Eamonn Sullivan So, you asked how I got on: I decided to try one of the pre-built images. The test/pull-request checks are working, but not the deploy. I get a TransferFailedException. I'm literally only hours old as a CircleCI user and I'm probably doing something basically wrong. #2020-10-27 15:50:43 practicalli Thanks for sharing #2020-10-23 11:23:15 alexmiller Yeah that doesn’t look right #2020-10-23 11:44:11 dominicm I also noticed that read-args uses a loop but could be mapv quite easily #2020-10-23 11:52:40 alexmiller fixed the double throw #2020-10-23 12:05:28 alexmiller agreed on read-args - that stuff has been through a lot of versions where the positionality was more important, going to leave it for now as it's reasonably likely to go through more changes #2020-10-23 14:21:41 dominicm @alexmiller I'm currently working on a main which will have exec-like uses, particularly the notion of updating a map in response to those args. Might be useful to rely on the exec code once it's stable. Then I can just point at your docs :p #2020-10-23 14:23:12 dominicm Obviously read-args is pretty simple, and so is the reduce itself. So maybe that's not really useful. But I'm not sure how it'll evolve. The tagged-literal reader might be subtle. #2020-10-23 14:23:43 alexmiller I would not rely on the exec code right now - I do not (yet) consider that to be "public api" #2020-10-23 14:24:09 alexmiller I think it would actually be useful to move some or all of it into clojure itself (similar to clojure.main) but that's probably longer term #2020-10-23 14:27:38 dominicm Yeah, I saw it was all private and ducked out :) #2020-10-23 11:09:43 arohner What is the recommended way to deploy a jar to a private S3 repo using tools.deps? #2020-10-23 14:08:52 dominicm This seems like an argument against exec-opts to me: clj -X:user/new create :template "\"$PWD/template::new-app\"" :name my.project.name (this doesn't work for me, but that's unrelated, just the awkward way to pass in pwd) #2020-10-23 14:17:14 alexmiller what doesn't work? #2020-10-23 14:18:18 alexmiller and it's awkward because? quoting? #2020-10-23 14:24:06 dominicm @alexmiller yeah, the quoting is awkward, compare to main-style:
--template "$PWD/template::new-app"
Doesn't work because I'm passing wrong args to clj-new, but I didn't need that debugging. #2020-10-23 16:50:02 seancorfield @dominicm an FYI on usability there: both clj-new and depstar ended up allowing symbols or strings in most argument slots because of quoting so :template foo works because it is silently converting the symbol to a string behind the scenes. But of course you can't use symbols in all cases (and particularly yours). #2020-10-23 16:50:59 dominicm @seancorfield heh, that feels like a dangerous path with spaces and backslashes, etc. #2020-10-23 16:51:23 dominicm I wonder if a directory named "4" would work, for example #2020-10-23 16:51:41 seancorfield Yup, it only works for "strings" that could be legal Clojure symbols... No, because that would be read as a number, not a symbol. #2020-10-23 16:51:42 dominicm Even my example is dangerous because it doesn't escape backslashes. #2020-10-23 16:52:05 dominicm Seems like you wouldn't want to script anything on that basis then. #2020-10-23 16:52:32 seancorfield #2020-10-23 16:56:27 seancorfield (because I just call str on it -- I thought I was being fancier than that) #2020-10-23 16:59:38 dominicm Str on a number works I guess? #2020-10-23 17:04:16 dominicm Anyway. Not a problem/solution situation really. Just an observation that this decision had this property.
I'm sure eventually we'll know when to apply each decision (or find a new decision) to get desired properties. #2020-10-23 17:13:22 seancorfield Yeah, that's why I crossed that out. I tried :4 as one of the args and got the equivalent of ":4" 🙂 Much depends on whether you want a plain string or, ultimately, a qualified symbol or relative path or... #2020-10-23 18:36:30 souenzzo I can't find how to do clj -m app.main in "modern #tools-deps #2020-10-23 18:38:59 Derek Passen clj -M -m app.main, I believe #2020-10-23 18:44:52 borkdude correct #2020-10-23 19:02:37 alexmiller just wait till we get to post-modern clj #2020-10-23 19:03:56 souenzzo can I stay using clj -m app.main on my deploy/scripts? #2020-10-23 19:09:16 alexmiller for now, yes (eventually that will stop working but not soon) #2020-10-23 19:06:22 borkdude in post-modern clj there is no one big story of arguments, but lots of different forks of clj where everyone does their own thing #2020-10-23 19:08:04 borkdude no one believes that hammock time eventually leads to the ultimate truth anymore. instead writing clojure programs becomes an expression of doubt #2020-10-23 19:08:19 borkdude :P #2020-10-23 19:08:40 dominicm I haven't released stable software since spec-ulation #2020-10-23 19:09:08 borkdude so hasn't core maybe? I should check #2020-10-23 19:09:27 alexmiller 1.10.1 :) #2020-10-23 19:10:43 drewverlee When i start an nrepl server through cider and eval datomic.client.api/client it complains about missing hmc auth. Indeed, if i check my cider-classpath that dep is missing.
The issue is the same as discussed here: https://clojurians-log.clojureverse.org/datomic/2020-07-22, which the solution being to update clojure. I updated via linuxbrew and now my version of clojure tools is 1.10.1.727. The issue remains.
My only idea is that if i ask cider to desribe the connection it says clojure is on 1.10.1 not 1.10.1.727 i'm not sure if its truncating or if it somehow connecting to an old version #2020-10-23 19:11:55 alexmiller 727 is the version of the clojure tools. the actual clojure version really is 1.10.1 #2020-10-23 19:12:31 alexmiller (unobviously, any version of the clojure tools can use any version of clojure - the clojure tools part really just tells you what it was built with and what version of clojure you get by default if you don't specify one) #2020-10-23 19:13:30 dominicm ❯ clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.0.0"}}}'
Downloading: org/clojure/clojure/1.0.0/clojure-1.0.0.pom from central
Downloading: org/clojure/clojure/1.0.0/clojure-1.0.0.jar from central
Clojure 1.0.0-
user=> *clojure-version*
{:major 1, :minor 0, :incremental 0, :qualifier ""}
user=>
#2020-10-23 19:16:26 dpsutton possible the erroneous classpath computed by an older tools deps is cached? just delete the cache? #2020-10-23 19:16:42 borkdude or use -Sforce #2020-10-23 19:18:30 drewverlee I deleted .cpcache (which i assume is the only cache) and it didn't work. I tried -Sforce, which starts a repl but then I can't connect to it with cider. I just get a "connection failed".
I'm going to try restarting my computer and assume this is a cider thing. Thanks for the suggestions. #2020-10-23 19:19:06 dpsutton its not CIDER. and restarting the computer won't clear the cache that is suspected to be at issue (if we are correct) #2020-10-23 19:21:37 alexmiller .cpcache is the classpath cache so sounds like you've done that already #2020-10-23 19:21:55 dpsutton oh wow. i totally glossed right over that message. sorry #2020-10-23 19:22:41 alexmiller oh that hmac auth thing was actually a tools deps bug and should be fixed in latest #2020-10-23 19:23:03 dominicm I wonder if there's 2 copies of clojure in $PATH or something... #2020-10-23 19:23:20 alexmiller yeah, can you repro outside cider? #2020-10-23 19:23:46 alexmiller if not, I'd guess you're somehow seeing a different version of clojure, or something is explicitly using an older version of tools.deps.alpha #2020-10-23 19:24:20 dominicm cider literally just runs clojure -Sdeps '{nrepl/nrepl … cider/cider-nrepl …}' -m nrepl.cmdline … #2020-10-23 19:25:09 drewverlee cider does just run clojure -Sdeps, but ill try using it directly just in case. #2020-10-23 19:26:18 dominicm cider's jack-in isn't magic, it's just saving you knowing which dep & which main to start. #2020-10-23 19:27:17 dpsutton (and now prints that and optionally the cljs startup to remove any semblance of magic) #2020-10-23 19:40:30 drewverlee The issue was corrected by having cider look at a version of clojure tools that had the bug fix. I had two versions of clojure installed (no useful reason for that). So i uninstalled both and reinstalled using linux brew.
As dominicm points out, you can look at what version of clojure cider is using in the message. It was different from what i got at the command line. So after verifying one version worked but not another, it was fairly obvious.
When i run clj /clojure from the command line it tells me the version of clojure, but in this case it seems it was the version of tools deps that was different between the two.
e.g /usr/local/bin/clojure starts a repl with "clojure 1.10.1" and the other version in /home/drewverlee/.linuxbrew/bin/clj says "clojure 1.10.1" so part of me assumed they were a symlink/alias. #2020-10-23 20:02:19 alexmiller cool #2020-10-24 13:21:44 dominicm {:aliases {:foo ["bar"]}} and clj -A:foo produces interesting results :P
e.g.
:resolve-args {\b \a},
:classpath-args {\b \a},
#2020-10-24 13:33:40 dominicm I was curious to see what would happen :) #2020-10-24 14:45:42 alexmiller The docs on the Clojure site always reflect the latest stable release (or use -h or the man page) #2020-10-24 15:10:39 holyjak Q1: Does :local/root need to be an absolute path? If so - I guess it isn't possible to use something like "~/path/to/it" (to make it portable across users)? #2020-10-24 15:13:38 holyjak Q2: Why am I getting
> Error building classpath. No coordinate type found for library cryogen-asciidoc/cryogen-asciidoc in coordinate {:local-root "/Users/me/external/cryogen-all/cryogen-asciidoc"}
when head /Users/me/external/cryogen-all/cryogen-asciidoc/pom.xml returns successfully (modified)
...
<groupId>cryogen-asciidoc</groupId>
<artifactId>cryogen-asciidoc</artifactId>
...
? #2020-10-24 15:18:48 alexmiller You can’t use ~ - that’s a shell thing #2020-10-24 15:19:00 alexmiller But either absolute or relative are fine #2020-10-24 15:21:05 alexmiller Not sure if that solves your q2 or not but if not share your deps.edn #2020-10-24 16:20:52 holyjak Here is it
{:deps {org.clojure/clojure {:mvn/version "1.10.1"}
ring-server/ring-server {:mvn/version "0.5.0"}
ring/ring-devel {:mvn/version "1.7.1"}
compojure/compojure {:mvn/version "1.6.1"}
cryogen-core/cryogen-core {:mvn/version "0.3.2"}
#_#_cryogen-core/cryogen-core {:local-root "../../external/cryogen-all/cryogen-core"}
#_#_cryogen-asciidoc/cryogen-asciidoc {:mvn/version "0.3.2"}
cryogen-asciidoc/cryogen-asciidoc {:local-root "../../external/cryogen-all/cryogen-asciidoc"}}
:aliases {;; Run with `clojure -M:build`
:build {:main-opts ["-m" "cryogen.core"]}
;; Start a server serving the blog: `clojure -X:serve`
;; (requires tools-deps 0.9.745+)
:serve {:exec-fn cryogen.server/serve
:exec-args {:port 8888}}}}
This is from https://github.com/holyjak/blog.jakubholy.net, the local dependency is https://github.com/cryogen-project/cryogen-asciidoc/ #2020-10-24 16:56:05 alexmiller It’s :local/root not :local-root #2020-10-24 17:00:20 holyjak OMG, thanks a lot! I must be blind. #2020-10-24 15:36:08 dpsutton I’m surprised you can’t use ~. I figured it would end up in a path object and the jvm would work it out for you #2020-10-24 15:40:05 borkdude jvm doesn't do this #2020-10-24 15:40:25 borkdude it's a bash thing. On Windows this doesn't work either #2020-10-24 15:40:26 dpsutton Makes sense. Not sure where I got that expectation #2020-10-24 15:41:20 borkdude What could work OS-independent is interpolation of environment variables. But even then you would have to use different names for linux and windows #2020-10-24 15:42:02 borkdude so probably a script that generates deps.edn for you as a pre-processing step is the least worst way of dealing with such concerns #2020-10-24 15:42:54 borkdude I haven't had an app where I did this, except for one where I keep deps in sync between project.clj and deps.edn #2020-10-24 15:50:41 alexmiller Java has a system property for this - user.home #2020-10-24 15:50:59 alexmiller (System/getProperty "user.home") is platform independent #2020-10-24 15:51:31 alexmiller but no equivalent to that in deps.edn (yet, or maybe ever) #2020-10-24 15:55:07 dominicm @holyjak what's the home-relative use case? #2020-10-24 16:13:49 holyjak None, if I can (I can) use relative psths #2020-10-24 16:21:45 dominicm ah, OK :) #2020-10-24 16:10:14 dharrigan I would love to see deps.edn support some type of user home substitution. I have the a main $HOME/.clojure directory, in there I have my deps.edn file, that I pull in other libraries with that are all local root (i.e., setting up a REPL with some middleware etc..). Right now, since I’m on Linux I have to put an absolute path in, i.e., :base {:extra-deps {local-base/local-base {:local/root "/home/david/.clojure/libs/local.base" :deps/manifest :deps}} #2020-10-24 16:10:26 dharrigan I cannot bring that across to my mac environment #2020-10-24 17:06:45 alexmiller Making the .clojure/deps.edn portable across machines wrt paths is not a goal - imo if you’re on a different machine, you will need to adjust to that machine #2020-10-24 17:07:51 alexmiller Which is not to say we won’t ever add something like this, but I’m not eager to do so #2020-10-24 16:10:47 dharrigan since that is /Users/home/david #2020-10-24 16:11:17 dharrigan I thus have to maintain two separate deps.edn files that are near identical, apart from the path stuff. #2020-10-24 16:12:57 dharrigan but and I’m just trying this out…having it as a relative path may work! #2020-10-24 16:13:15 dharrigan i.e., :base {:extra-deps {local-base/local-base {:local/root "libs/local.base" :deps/manifest :deps}} #2020-10-24 16:14:07 dharrigan nope #2020-10-24 16:14:23 dharrigan Switching to a relative path, ends up with #2020-10-24 16:14:26 dharrigan Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate local/rebel/main__init.class, local/rebel/main.clj or local/rebel/main.cljc on classpath.
#2020-10-24 16:15:22 dharrigan which leaves me with having to put the absolute path in my deps.edn, and maintaining two separate deps.edn (which I suspect, if I was also on windows would incur more mangement) #2020-10-24 16:18:05 dharrigan It would be nice IMHO if deps supported at least having :local/root resolve the user’s home. #2020-10-24 16:27:07 dominicm Hmm, I'm slightly surprised that resolution isn't relative when doing that. The personal tooling case is the only one I have so far too :) #2020-10-24 16:57:57 alexmiller I’m not, that’s what I’d expect - top level deps are relative to where you’re running clj #2020-10-24 16:59:06 dominicm @alexmiller I suppose they could be resolved with a relative *dir* like when doing transitive :local/root though? #2020-10-24 16:59:14 dominicm (obviously there's problems there due to the merging) #2020-10-24 16:59:36 alexmiller Yeah, that’s problematic #2020-10-24 17:10:55 dominicm I suppose paths could be canonicalized pre-merging. #2020-10-24 18:13:51 borkdude that's what I do in clj-kondo config. Each config can refer to local dirs relative to itself and these get resolved during config reading. #2020-10-24 18:14:49 borkdude this way you can have hooks in your home dir config, not relative to your project #2020-10-24 18:44:00 alexmiller That’s possible, I’ll think about it #2020-10-24 18:59:38 dharrigan that would be nice #2020-10-24 18:59:39 dharrigan 🙂 #2020-10-26 08:12:01 onetom i have an alias defined in my deps.edn as follows:
:ion-dev
{:extra-deps {com.datomic/ion-dev {:mvn/version "0.9.276"}}
:main-opts ["-m" "datomic.ion.dev"]}}
if i run /nix/store/6n60jkgpxb1a95wjx3ikhn9nvp5w33yh-clojure-1.10.1.727/bin/clojure -A:ion-dev -Sdeps '{:deps {nrepl {:mvn/version "0.8.0"}...' -m nrepl.cmdline ... it complains
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
WARNING: When invoking clojure.main, use -M
does this mean
1. i have to move the :main-opts into a different alias?
2. the command line should have a -M (without any aliases as a parameter) if i just want to run clojure.main? #2020-10-26 08:16:18 onetom im wondering what should be the naming convention for aliases in cases when one want to use some extra dependency both as just a library and as a command-line tool...
-A:ion-dev -M:ion-dev/run maybe? #2020-10-26 08:40:29 onetom am i supposed to combine aliases which doesn't contain :main-opts with aliases which does?
like -M:ion-dev:ion-dev/run?
when my deps.edn has:
:ion-dev
{:extra-deps {com.datomic/ion-dev {:mvn/version "0.9.276"}}}
:ion-dev/run
{:main-opts ["-m" "datomic.ion.dev"]}} #2020-10-26 08:46:58 dominicm @onetom needs a -M before the -m, so clojure -A:ion-dev -Sdeps … -M -m nrepl.cmdline … #2020-10-26 10:29:06 practicalli @onetom it is not required to have separate aliases, you can use the original :ion-dev alias and just use -M instead of -A . However, the -M (and -A flags) should come after the -Sdeps option and before the -m option (the -A flag does still work out of position though)
clojure -Sdeps '{:deps {nrepl {:mvn/version "0.8.0"}...' -M:ion-dev -m nrepl.cmdline
It is also possible to chain the aliases under -M , especially if there is an alias for nrepl too (this example taken from https://github.com/practicalli/clojure-deps-edn/)
:middleware/nrepl
{:extra-deps {nrepl/nrepl {:mvn/version "0.8.2"}}
:main-opts ["-m" "nrepl.cmdline"]}
And then simplify the command
clojure -M:ion-dev:middleware/nrepl
#2020-10-26 12:02:13 onetom sure, but i need the :extra-path in :ion-dev on its own too, without the :main-opts and so far i haven't found any option combination which wouldn't print some kind of warnings. #2020-10-26 12:55:46 practicalli :replace-deps will just use the deps specified in the alias, dropping any libraries from the main :deps section. This is useful if you just want to run an external tool, e.g. such as clj-new that creates a new Clojure project.
Using the -M option only calls the main namespace from the last alias in a chain, so with -M:ion-dev:middleware/nrepl only the main namespace from :middleware/nrep is called and the :ion-dev is simply adding the library as an additional dependency.
I didnt see any other main namespaces on the example shared.
I am not clear on why an :icon-dev with only extra paths is required in the scenario shared so far. #2020-10-26 13:07:12 onetom to start a development repl (with cursive or cider) #2020-10-26 13:08:24 onetom but i want to retain the option of running datomic.ion.dev/-main, since that's the official way to deploy datomic cloud projects. #2020-10-26 13:09:56 onetom when i have a repl running already (started with repl related :main-opts), im just using datomic.ion.dev/{push,deploy,deploy-status} directly #2020-10-26 13:33:33 alexmiller There is no option to use the extra-deps but not the main-opts from an alias #2020-10-26 13:33:53 alexmiller You should separate those into two separate aliases if you need that #2020-10-26 14:09:55 arohner Something in my deps is using a version range or SNAPSHOT because every time I run clj, it runs
Downloading: com/amazonaws/aws-java-sdk-cloudwatch/maven-metadata.xml
. Is there a clj command that will help me find which dep that is? #2020-10-26 14:11:41 arohner it’s not -Strace or -Stree, because those only tell me the resolved version #2020-10-26 14:16:12 alexmiller if you do tree, it's probably whatever is above that dep in the tree ? #2020-10-26 14:17:32 alexmiller I don't think I have any other magic command to tell you more unfortunately #2020-10-26 14:20:47 arohner that helps some, thanks #2020-10-26 14:21:37 arohner I was able to find the dep depending on cloudwatch (amazonica), but something else is causing
Downloading: org/clojure/clojure/maven-metadata.xml
. I guess I can wipe out ~/.m2/repository and do some grepping #2020-10-26 14:22:18 alexmiller alternately, use -Sdeps '{:mvn/local-repo "tmp"}' #2020-10-26 14:22:30 alexmiller build up a new tmp repository instead #2020-10-26 15:58:44 arohner found it. de.kotka/lazymap specifies:
<dependency>
<groupId>org.clojure</groupId>
<artifactId>clojure</artifactId>
<version>[1.2,1.6)</version>
<scope>compile</scope>
</dependency> #2020-10-26 15:59:04 arohner should that always trigger a snapshot check? #2020-10-26 16:04:15 alexmiller all version ranges will trigger a resolution to the "newest" stable version #2020-10-26 16:05:07 arohner even when bounded like that? #2020-10-26 16:05:18 alexmiller yes #2020-10-26 16:06:01 alexmiller you could :exclude org.clojure/clojure down that path #2020-10-26 16:06:27 alexmiller although based on where the canonicalization happens, that may not help #2020-10-26 16:11:26 alexmiller that may be underdocumented :) #2020-10-26 16:11:35 alexmiller :exclusions [org.clojure/clojure] #2020-10-26 16:11:39 alexmiller in the coordinate #2020-10-26 16:11:52 alexmiller it looks like lazymap 3.1.1 dropped the version range #2020-10-26 16:14:32 alexmiller I don't think exclusions will help you here #2020-10-26 16:15:12 alexmiller at the point where expansion goes to resolve the deps, it's resolving all of them, the exclusions and stuff are handled at a higher level #2020-10-26 16:15:27 arohner Putting lazymap 3.1.1 as an explicit dependency fixed it for me #2020-10-26 16:15:34 alexmiller yeah, that should work #2020-10-26 16:15:55 arohner lazymap isn’t a dep of my project, it’s a dep of a dep. Putting lazymap in :override-deps doesn’t seem to do anything. Is it supposed to? #2020-10-26 16:39:28 seancorfield @arohner :override-deps only works inside an alias #2020-10-26 17:02:54 arohner thanks #2020-10-27 11:07:43 mkvlr Hey 👋 We’d like to bring in modules from another private repo using gitlibs. Ideally we’d want to use ssh auth. There’s no way to add support for another coordinate or get it to use tools.gitlibs from https://github.com/ghadishayban/tools.gitlibs/pull/1 without forking tools.deps (and having to distribute it to everybody on the team), is there? Or any other solutions to this problem that folks can recommend? Thanks! #2020-10-27 11:11:22 borkdude If all else fails, you could manually pull the code and add it via local/root? #2020-10-27 11:14:15 mkvlr yep, guess that’s one option. Was also looking at your https://github.com/borkdude/deps.clj since we do require babashka on the project but the compute classpath part still happens on the jvm using the normal t.d.a jars right? #2020-10-27 11:14:34 borkdude correct #2020-10-27 11:16:47 borkdude it pulls tools jars that are exactly the same as the jars that clj uses. however, we could make that configurable ;) #2020-10-27 11:17:29 borkdude but maybe best to not create forks of official tools when it can be fixed in those official tools #2020-10-27 11:18:00 mkvlr right #2020-10-27 14:06:31 alexmiller you can compile the java source into java class files, then add the path containing those to your paths #2020-10-27 14:10:19 kirill.salykin i see, thanks for the options #2020-10-27 14:29:32 borkdude would it be an idea to extend -X with being able to pass a single map?
clj -X:graph graph :deps '"mydeps.edn"' :trace true :output '"trace"'
clj -X:graph graph '{:deps "mydeps.edn" :trace true :output "trace"}'
The quoting of strings seems a little clumsy on the command line, so quoting the entire map can deal with that as a whole. #2020-10-27 14:30:41 borkdude I've been adopting the latter style for more tools where I just pass the entire map as --opts, e.g.:
$ echo '[1 2 3]' | puget --opts '{:print-color false}' #2020-10-27 14:31:44 borkdude Another question: is the -X CLI parsing logic available as a library? I think this could be used in other places than clj as well (e.g. GraalVM compiled binaries). #2020-10-27 14:38:58 alexmiller answering the last question first, no but that's something we're actively thinking about - I think it's something that could end up in Clojure itself (like clojure.main) but I am still actively changing the interface to it and will need further work before I think it's there #2020-10-27 14:40:18 alexmiller on the former question, that's a good question, will think about it #2020-10-27 14:41:30 alexmiller I mean really the idea was to put the base map in alias data and then just overrides on the cli, but some tools would benefit more than others from this #2020-10-27 21:13:05 dominicm I keep thinking I've missed a way to have some kind of :deps-aliases in an alias to use the extra deps from another alias, similarly to how I can with :extra-paths. #2020-10-27 21:16:50 alexmiller I'm confused by what you mean about :extra-paths #2020-10-27 21:17:37 alexmiller unless you mean paths as alias data #2020-10-27 21:17:46 dominicm I mean that I'm surprised there's no equivalent to :extra-paths [:build-paths] such as :extra-deps [:build-deps]. Obviously that syntax doesn't work, but that general idea. #2020-10-27 21:17:48 dominicm I do yeah :) #2020-10-27 21:17:52 alexmiller ah, ok #2020-10-27 21:18:32 alexmiller would certainly complicate the (already too complicated) merging logic #2020-10-27 21:19:00 dominicm :extra-deps [:build-deps {…}] maybe… #2020-10-27 21:21:27 alexmiller I'd probably be tempted to define something mappy instead so the merging would continue to work (like destructuring syntax or spec selects) #2020-10-27 21:22:10 dominicm :extra-deps {:clojure.tools.deps/aliases [:build-deps] my.cool/lib {}} ? #2020-10-27 21:22:27 alexmiller well, I'm not going to design it right now :) #2020-10-27 21:22:41 alexmiller if you wanted to file a jira or an ask request, would be glad to keep that idea around #2020-10-27 21:25:58 dominicm I'll open a JIRA. :) #2020-10-27 21:26:15 alexmiller thx #2020-10-29 15:41:45 dpsutton working to give an error message to verify that the tools deps version is too old to support -M -m. Does anyone know offhand if a version supported -X but not -M -m? Hoping can check something real quick with -X and that means it necessarily supports -M #2020-10-29 15:44:24 dpsutton as it stands, added a simple (ns check) (defn -main [& _] (System/exit 0)) but would prefer to not need a new main and new ns if X and M go hand in hand #2020-10-29 16:27:38 dpsutton ie, could just do clojure -X clojure.core/identity :a :a and if that returns 0 then good to go for -M -m our.stuff #2020-10-29 16:42:41 seancorfield -M has been in the CLI for a long time but it's meaning changed around 1.10.1.697 #2020-10-29 16:43:45 seancorfield The difference between 1.10.1.561 (stable) and the later version is just that -M now respects resolve-args and classpath-args in aliases as well as main-opts. Previously, it ignored resolve-args and classpath-args. Does that help @dpsutton? #2020-10-29 16:45:30 dpsutton Yeah that does. I think that means we should use the check for X support to make sure we have a recent enough version #2020-10-29 16:45:59 dpsutton Although I think our use case is simple enough that as long as M support is there it’s fine #2020-10-29 16:59:22 seancorfield @dpsutton I just ran some tests on a bunch of prerelease versions between the last two stable releases, and there are several where that command returns a zero status but doesn't actually work. #2020-10-29 17:00:26 seancorfield (but I believe they would support -M with aliases by that point... I just haven't tested that 🙂 ) #2020-10-29 17:02:00 dpsutton In what way do they not work? #2020-10-29 17:02:09 dpsutton And thanks so much! #2020-10-29 17:02:40 seancorfield (! 1626)-> /usr/local/Cellar/clojure\@1.10.1.636/1.10.1.636/bin/clojure -X clojure.core/identity :a :a
No alias specified with -X
Thu Oct 29 09:58:15
(sean)-(jobs:0)-(/Developer/workspace/wsmain/build)
(! 1627)-> echo $?
0
#2020-10-29 17:03:13 seancorfield The ability to pass the fn to execute on the command-line came fairly late in that set of prerelease builds. #2020-10-29 17:04:28 seancorfield 1.10.1.672 seems to be the first prerelease build that allowed it. #2020-10-29 17:05:18 dpsutton Oh ok. It’s only prereleases? If so they can diagnose it on their own :) #2020-10-29 17:05:40 dpsutton Bleeding edge is sometimes bleeding #2020-10-29 17:06:13 seancorfield Yeah, 1.10.1.561 (previous stable) throws an exception and gives a non-zero exit status. #2020-10-29 17:06:31 seancorfield (so does 1.10.1.590 which was the first of the prerelease builds) #2020-10-29 17:06:40 dpsutton Beautiful. How do you change versions so quickly? Would have done that if I knew a good way to swap them around easily #2020-10-29 17:07:46 seancorfield I installed them by version with brew and then I can use the full path (as shown above) to run any version I want 🙂 #2020-10-29 17:08:29 kirill.salykin how are you doing this? #2020-10-29 17:09:01 kirill.salykin just brew install @version ? #2020-10-29 17:09:09 seancorfield brew install clojure/tools/ per the CLI install docs #2020-10-29 17:09:31 kirill.salykin clear, thanks a lot! #2020-10-29 17:07:58 seancorfield {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "2a494645405f584f6a1b041b1a041b041f1c1b"}, :content ("[email protected]")} #2020-10-29 17:08:34 seancorfield I didn't have 561 until just now, but installed it by version to test. #2020-10-29 17:14:30 dpsutton Oh I didn’t know it left older versions laying around #2020-10-30 21:04:51 borkdude clojure -A:foo and clojure -M:foo without main-opts do the same thing: invoke a REPL with alias foo merged in -- right? Should -M:foo give a warning when you forget to supply main-opts in deps.edn maybe? #2020-10-30 21:05:30 borkdude or should -A not invoke a REPL since that's a clojure main invocation? #2020-10-30 21:15:32 seancorfield -A is for REPL invocation, according to the docs. #2020-10-30 21:16:37 borkdude so -A and -M overlap there I guess #2020-10-30 21:17:16 seancorfield I suspect that's a bit of a grey area given that :main-opts ["-r"] works to run a REPL, yes? #2020-10-30 21:17:42 borkdude well, I invoked an alias with -M while I forgot to provide :main-opts in deps.edn, so it invoked a REPL for me #2020-10-30 21:17:57 borkdude that's what prompted these maybe too far fetched thoughts #2020-10-30 21:18:51 seancorfield Yeah, I would sort of expect -M to at least give a warning if no :main-opts are provided. @alexmiller? #2020-10-30 21:19:54 seancorfield (! 651)-> fgrep :repl deps.edn
:repl {:main-opts ["-r"]}
(! 652)-> clj -M:repl
Clojure 1.10.1
user=> ^D
(! 653)-> clj -A:repl
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.
Clojure 1.10.1
user=> ^D #2020-10-30 21:20:22 seancorfield (so, yeah, you can specify -r explicitly as a :main-opts item) #2020-10-30 21:21:49 seancorfield And the corollary:
(! 656)-> fgrep :dummy deps.edn
:dummy {}
(! 657)-> clj -M:dummy
Clojure 1.10.1
user=> ^D
(! 658)-> clj -A:dummy
Clojure 1.10.1
user=> ^D #2020-10-30 21:22:20 alexmiller you can supply main-opts on the command line, so -M does necessarily have to have :main-opts #2020-10-30 21:22:56 alexmiller clj -M:my-alias-with-just-deps -m my.ns #2020-10-30 21:23:09 seancorfield Yes, but that's not the case we're discussing here. #2020-10-30 21:23:26 alexmiller well, what is? #2020-10-30 21:23:50 seancorfield See the two console examples I pasted. No command-line main-opts there. #2020-10-30 21:23:56 alexmiller -M invokes clojure.main. clojure.main does not require opts. #2020-10-30 21:25:23 seancorfield Fair enough. The "implicit" -r option 🙂 #2020-10-30 21:26:11 seancorfield Why draw the distinction in the docs then that -A is for REPL execution? #2020-10-30 21:26:30 borkdude because -M might not invoke a REPL when main-opts are given #2020-10-30 21:26:45 alexmiller really, it can be used to supply aliases for anything but it's only useful with REPL (because -M can be used for main and -X can be used for functions) #2020-10-30 21:27:18 alexmiller (and for allowing legacy stuff to continue working) #2020-10-30 21:27:20 seancorfield I guess this will become less confusing once -A stops executing :main-opts #2020-10-30 21:27:33 alexmiller yes #2020-10-30 21:28:16 seancorfield I'm curious how you plan to distinguish between -A for a REPL and -M executing clojure.main which might pick up other main opts? #2020-10-30 21:30:07 alexmiller what do I need to distinguish? #2020-10-30 21:31:02 seancorfield I mean: how is -A going to start a REPL without running clojure.main? #2020-10-30 21:31:33 alexmiller it already does this #2020-10-30 21:31:43 alexmiller it invokes clojure.main #2020-10-30 21:31:47 alexmiller that's an implementation detail #2020-10-30 21:32:28 alexmiller clj without -M or -X will start a repl, you let me figure out how to do that :) #2020-10-30 21:33:00 seancorfield So, in the absence of :main-opts in an alias, both -A and -M will work identically, always? i.e., they will both run clojure.main and they will both start a REPL? #2020-10-30 21:28:19 borkdude I would maybe prefer -M to trigger a warning when there is no :main-opts [] when you want to explicitly pass 0 arguments to main (i.e. invoke a REPL) #2020-10-30 21:29:13 seancorfield @borkdude I don't think that is going in the right direction... #2020-10-30 21:30:35 seancorfield It seems that the only way to avoid continued confusion is to have clojure.main without arguments not start a REPL -- and require -r for that? 🙂 #2020-10-30 21:30:55 alexmiller I don't follow #2020-10-30 21:31:19 borkdude never mind #2020-10-30 21:33:56 alexmiller yes? #2020-10-30 21:34:43 seancorfield I guess you could always have a new entry point for a REPL and have -A invoke that...? #2020-10-30 21:35:16 alexmiller I don't understand what problem is bothering you #2020-10-30 21:35:16 seancorfield (or rather "have the absence of -M and -X invoke that") #2020-10-30 21:35:41 alexmiller the absence of -M or -X means: invoke a repl #2020-10-30 21:36:09 alexmiller in other words, the default behavior of clj is: invoke a repl #2020-10-30 21:36:09 seancorfield (! 659)-> clj -A:dummy -r
WARNING: When invoking clojure.main, use -M
Clojure 1.10.1
user=> ^D
#2020-10-30 21:36:53 alexmiller ok, you're outside the bounds of documented behavior #2020-10-30 21:37:13 alexmiller that's legacy support #2020-10-30 21:37:19 seancorfield -r is a documented main-opt that causes clojure.main to start a REPL, isn't it? #2020-10-30 21:37:46 alexmiller yes, but when we stop accepting main opts outside of -M, that's no longer a thing #2020-10-30 21:38:10 alexmiller there may be some other set of accepted arguments in the repl case that have nothing to do with clojure.main #2020-10-30 21:38:13 seancorfield OK, so there will need to be a change in the behavior of clojure.main at some point to "fix" this? #2020-10-30 21:38:18 alexmiller no #2020-10-30 21:38:29 seancorfield clojure.main's help says
-r, --repl Run a repl
#2020-10-30 21:38:29 alexmiller "clj starting a repl" is distinct from clojure.main #2020-10-30 21:39:01 alexmiller yes, the mechanism clj currently uses to start a repl is clojure.main #2020-10-30 21:39:07 alexmiller but that's an implementation detail #2020-10-30 21:39:40 seancorfield So there will be a non-`clojure.main` entry point that clj can use to start a REPL? #2020-10-30 21:39:53 seancorfield (otherwise this confusion is going to continue) #2020-10-30 21:40:05 alexmiller what I'm saying is - I am promising clj will start a repl. the end. #2020-10-30 21:40:34 alexmiller and I'm promising that clj -M will run clojure.main #2020-10-30 21:40:55 alexmiller and that clj -X will invoke a function with a map #2020-10-30 21:41:05 seancorfield but you're not promising that clojure.main will not support -r at some future point :rolling_on_the_floor_laughing: #2020-10-30 21:41:32 alexmiller there are no plans to make clojure.main do anything different than what it does now #2020-10-30 21:42:52 alexmiller in the before times, clj always ran clojure.main. that is no longer the model #2020-10-30 21:43:22 alexmiller the new model is:
• by default, run a repl
• with -M, run clojure.main
• with -X, execute a function with a map #2020-10-30 21:43:40 alexmiller and "run a repl" does not say anything about clojure.main #2020-10-30 21:45:29 alexmiller if you want to explicitly run clojure.main to run a repl, then clj -M or clj -M -r will do that #2020-10-30 21:45:44 seancorfield The confusion arises because the docs talk about "using -A (for REPL invocation)" when it still runs :main-opts! #2020-10-30 21:46:17 alexmiller well, we're in a bridge period between the two models #2020-10-30 21:46:31 seancorfield The sooner -A stops running :main-opts the better... #2020-10-30 21:46:58 seancorfield (mind you, even in that future world, I think @borkdude’s confusion will still be present, yes?) #2020-10-30 21:47:40 alexmiller if the confusion is that you can use clojure.main to run a repl, then yes #2020-10-30 21:48:40 alexmiller but it's easy to reason about if we are properly conveying that -M == run clojure.main #2020-10-30 21:48:45 seancorfield Argh, I'm being bitten by caching here 😞 #2020-10-30 21:49:01 seancorfield (! 665)-> clj -Sforce -A
Missing required argument for "-A ALIASES"
(! 666)-> clj -Sforce -A -r
Missing required argument for "-A ALIASES"
#2020-10-30 21:49:08 seancorfield ^ That makes more sense. #2020-10-30 21:49:46 seancorfield -M and -X can be invoked without an alias, -A cannot. #2020-10-30 21:49:49 alexmiller what does that have to do with caching? -A always requires an alias #2020-10-30 21:50:26 seancorfield Because this happened before I added -Sforce:
(! 662)-> clj -A
Clojure 1.10.1
user=>
(! 663)-> clj -A -r
WARNING: When invoking clojure.main, use -M
Clojure 1.10.1
user=> ^D
#2020-10-30 21:50:59 seancorfield (note the command history numbers there!) #2020-10-30 21:51:10 alexmiller oh, well that's just a bug then #2020-10-30 21:54:46 seancorfield Here's the repro for that bug:
(! 674)-> clj -Sforce -A
Missing required argument for "-A ALIASES"
(! 675)-> clj -A
Clojure 1.10.1
user=>
#2020-10-30 21:55:38 alexmiller fixed #2020-10-30 21:56:09 borkdude So the point earlier was: invoking -M with an alias that doesn't have :main-opts assumes that :main-opts is implicitly []. My question was: should this be less implicit, to avoid people ending up in a REPL when invoking -M with an alias that wasn't intended for -M at all #2020-10-30 21:57:16 alexmiller no, because :main-opts is not required #2020-10-30 21:57:28 borkdude ok #2020-10-30 21:58:17 seancorfield @borkdude in the following :test has no :main-opts -- which of these would you expect to get warnings on and why?
(! 676)-> clj -Sforce -M:test -r
Clojure 1.10.1
user=> ^D
(! 677)-> clj -Sforce -A:test -r
WARNING: When invoking clojure.main, use -M
Clojure 1.10.1
user=> ^D
(! 678)-> clj -Sforce -M:test
Clojure 1.10.1
user=> ^D
(! 679)-> clj -Sforce -A:test
Clojure 1.10.1
user=> ^D
#2020-10-30 21:58:20 alexmiller we could do something like - if you supplied -M but did not supply either :main-opts or main-opts on the command line, then either warn or error at that point #2020-10-30 21:58:51 alexmiller but honestly that's difficult to check right now due to where that knowledge of different parts of that is #2020-10-30 21:58:53 borkdude yes, that's exactly the case I was referring to #2020-10-30 21:58:54 seancorfield So -M wouldn't silently start a REPL unless you said -M -r ? I think that would be reasonable. #2020-10-30 21:59:08 seancorfield (if it could be checked) #2020-10-30 21:59:23 alexmiller and then -M means run clojure.main except for this one case where it fails #2020-10-30 21:59:42 borkdude yeah, I guess it's not worth changing #2020-10-30 21:59:57 seancorfield (I'd be against an error there but I think a warning to remind folks might be worthwhile) #2020-10-30 22:00:59 borkdude main opts can be supplied on the command line, so not providing arguments means the same as :main-opts [], but you can also do that with -A which is the preferred way of invoking a REPL maybe #2020-10-30 22:01:53 alexmiller yes #2020-10-30 22:01:11 alexmiller @seancorfield 1.10.1.735 if you want to check the -A thing #2020-10-30 22:01:17 seancorfield But I suspect there's no point trying to second-guess what folks will find confusing until after we have a CLI version where -A no longer uses :main-opts #2020-10-30 22:01:31 seancorfield @alexmiller Thanks for the swift fix! #2020-10-30 22:03:45 seancorfield (! 680)-> clj -Sforce -A
-A requires an alias
(! 681)-> clj -A
-A requires an alias
confirmed fixed! #2020-10-30 22:04:13 alexmiller I did not take the time to try on Windows, but it might already catch this case due to the different parsing. I'll check that before we get to a stable release #2020-10-30 22:04:46 alexmiller I'm shutting down, have a good Halloween weekend y'all #2020-11-01 18:04:58 practicalli You need to define an alias to use with the -X flag.
My understanding is the -X flag should be used with an alias, as I havent seen any examples of using :exec-fn as a top level key. From the error you are getting, this seems to be a correct...
There are plenty of examples of :exec-fn , :exec-args and :ns-default in the practicalli/clojure-deps-edn
https://github.com/practicalli/clojure-deps-edn #2020-11-01 18:05:08 vlaaad I think exec-fn has to be in alias, and -X requires alias #2020-11-01 18:06:01 practicalli snap 🙂 #2020-11-01 18:07:27 practicalli You can specify :exec-fn on the command line I believe, if you really dont want an alias
Edit: hmm, I am not sure that is right now. You can specify a function, but still has to be with an alias.
Edit: as borkdues says below, if its a fully qualified function on the class path, then the command line is enough - need to add some good examples to the book so I dont forget #2020-11-01 19:00:52 borkdude That's not true though:
$ clj -X clojure.core/prn :foo 1 :bar 2
{:foo 1, :bar 2} #2020-11-01 19:28:05 seancorfield You can't run just clj -X on its own. You either have to provide an alias or you have to specify the function to call on the command-line. #2020-11-01 19:28:50 seancorfield Just like :main-opts can't be at the top level in deps.edn, :exec-fn, :exec-args, :ns-default, etc all need to be under an alias. #2020-11-01 19:31:34 seancorfield There are only a few keys valid at the top level @U0522TWDA
:deps - map of lib (symbol) to coordinate
:paths - vector of paths
:aliases - map of alias name to alias data
provider-specific keys for configuring dependency sources
(that last one is stuff like :mvn/repos) #2020-11-01 19:39:10 holyjak Thanks a lot! #2020-11-02 14:27:54 alexmiller have been through a few rounds on the patch, but I haven't looked at it recently #2020-11-02 22:23:49 olivergeorge Thanks Alex #2020-11-02 04:27:20 olivergeorge #2020-11-02 11:32:46 borkdude Got part 1 working of clj-kondo deps.edn linting: #2020-11-02 22:21:29 alexmiller when you look at a deps tree (`clj -Stree` , lein deps :tree , mvn dependency:tree ), what question are you trying to answer?
1. what are the set of deps I'm using?
2. what version of a particular dep am I using?
3. what are the transitive deps of a particular library?
4. is a particular dep version out of date?
5. is some dep I'm expecting not on my classpath?
6. why is some dep on my classpath? #2020-11-02 22:33:59 andy.fingerhut You should give higher weight to the answers of others than me (I'm not using Clojure in production myself), but usually 1 3 6 and sometimes 2. #2020-11-02 22:34:36 andy.fingerhut Seems like a question you could reasonably put on State of Clojure Survey 2021 if you were so inclined 🙂 #2020-11-02 22:49:04 alexmiller so, interestingly you can't answer #3 or to some extent #6 correctly based on the result of any of those tools #2020-11-02 22:51:25 alexmiller in many cases deps show up at multiple places in the "tree" (really, a graph) and all print it only at one location (trivial example, clojure is usually included N times but only shows up once) #2020-11-02 23:14:59 nate I don't know if this falls into 5 or 6, but I most often: what is forcing dep x to be version y? #2020-11-02 23:15:26 nate sometimes I use it for 1, but more often that's -Spath with a grep to find a particular dep #2020-11-02 23:37:22 dorab Mostly 6 and 2 for me. 5 and 3 sometimes. #2020-11-03 07:25:00 dharrigan 6 3 5 1 and lesser 4 2 since I use antq regularly to ensure my deps are up-to-date. #2020-11-03 08:02:43 vlaaad I guess that might be a variation of 6, but “what deps depend on this dep?” #2020-11-03 08:40:23 dominicm 1 2 5 6. With the additional use case of chasing "which dependencies transitively depend on A, and need to have exclusions added." That's particularly annoying in clj as it only shows the dependency that was retained. #2020-11-03 09:14:32 rickmoynihan I’d say all of them to some extent, but mainly 5, 6 and 2…
1 is more often asked as a general high-level question; so I’d normally just look at the deps.edn to get a rough overview, and overview of the intention.
I usually use -Stree etc to help answer a slightly different question, which is “What deps are being chosen and why”. This usually being a combination of the above options. I think the distinction is often when I’m running -Stree it’s as a debugging tool, and I don’t know what the problem is yet (i.e. it could be 1 because I’ve started with the wrong aliases, it could be 2 because the wrong version of a transitive dep is being selected, it could be 3 if we’re accidentally explicitly depending on a transitive dep which has been removed or isn’t available in the alias combination. Similarly for 5 sometimes 6 if an unused dep is causing a transitive dep to be a different version).
For example all of those questions might uncover the reason behind a ClassNotFoundException, or perhaps a method being called with the wrong arguments etc. But until you run -Stree you don’t know which it’s going to be. #2020-11-03 09:17:50 rickmoynihan Ahh just seen @U04V70XH6 said essentially the same thing in the main channel #2020-11-02 22:24:59 alexmiller some possible answers there, but that's an open set #2020-11-02 22:52:48 dpsutton wanting other use cases or to chime in with which of the above are most useful? #2020-11-02 22:53:18 alexmiller just what it says above - when you reach for this, what question are you usually trying to answer? #2020-11-02 22:54:13 alexmiller or separately, what are some dep tree related questions you would like to be able to answer? #2020-11-02 22:55:17 dpsutton used it today to diagnose a bad release of piggieback. it had some extra junk tucked in the jar and the cleanest repo was to clj -Sdeps '{:deps {cider/piggieback {:mvn/version "0.5.1"}}}' -Stree. then rebuilt the jar locally and checked with clj -Sdeps '{:deps {cider/piggieback {:local/root "target/piggieback-0.5.1.jar"}}}' -Stree that the resultant jar was clean so it was just a deploy problem #2020-11-02 22:56:10 alexmiller so this is really #1 in a diff scenario #2020-11-02 22:56:36 dpsutton yes. 1 and 3 #2020-11-02 22:58:01 alexmiller well it doesn't really answer #3 - that's often a lie as it's thinning a graph into a tree #2020-11-02 22:55:20 ghadi Usually 1 & 3 for me #2020-11-02 23:01:08 seancorfield My two scenarios are: 1. trying to debug a problem that turns out to either be an unexpected transitive dependency or an unexpected version of one of them (which is a combination of several of those six things to one degree or another) and 2. as part of my script that attempts to determine #4 above (by synthesizing a new project from the output of clojure -Stree and then running clojure -Stree on that and seeing what versions changed -- but only because none of the "outdated deps" tools out there work properly in our monorepo setup at work). #2020-11-02 23:03:13 seancorfield I'd say scenario 1 is a mix of #3, #5, and #6 primarily (and I realize that the tree isn't a full/accurate representation of the actual graph, but it's "close enough" and can always be used selectively with -Sdeps to select specific top-level deps to examine further). #2020-11-04 18:25:31 rickmoynihan So I figure this might be of some interest to people on this channel.
I’ve forked @slipset’s deps-deploy to support a number of new features and have created PRs for each of them:
1. Configuration via :exec-args and all that means (i.e. overriding parameters via the CLI) https://github.com/slipset/deps-deploy/pull/18
2. Support for private S3 buckets via s3-wagon https://github.com/slipset/deps-deploy/pull/19
3. A new feature to allow overriding the version tag in the pom.xml with whatever you supply as a :mvn/version in :exec-args or more commonly through a command line override. My hope is this should help generating new artifacts with new versions in CI. https://github.com/slipset/deps-deploy/pull/20
I figured I’d mention this in case anyone has any comments or feedback here.
It’s yet to be documented, and slipset is yet to agree to any of it, and there are some other things I’d like to add too like the ability to influence more of the parameters that are passed into pomegranate/aether (which would allow for example configuration of proxies etc in deps.edn), but I thought it might be worth mentioning. #2020-11-04 19:17:04 dominicm The other common override along with version is the scm tag (which is usually v${version}). #2020-11-04 21:38:25 rickmoynihan What are the usecases for overriding it? #2020-11-04 21:56:11 dominicm The tag points at the commit or git tag with the release pertains to. So you should override it when releasing so, e.g. Cljdoc will pick up the write commit to read docs from. #2020-11-04 22:00:29 rickmoynihan thanks makes sense #2020-11-04 22:05:20 dominicm Totally going to move away from garamond to your thing. #2020-11-05 17:34:47 rickmoynihan I wonder if the override feature might be better placed in depstar?! #2020-11-05 17:57:07 dominicm I think there's a lot of ambiguity in pom processing right now #2020-11-06 09:17:54 rickmoynihan yeah that’s precisely my point…
Personally I feel like deps-deploy and depstar should probably be integrated into the same project. And use each others :exec-args keys etc..
Though I guess that depends on whether there are any plans for the cli tools to ever support task composition. (I know it’s been raised here a few times). Here’s hoping tools.build lets you compose tasks intelligently :crossed_fingers:
The project I’m working on wants to build and deploy multiple builds (essentially themed versions of the same thing), and to do that right now I need to fire up two JVMs for each build — and we might need 10 builds at some point… So that’ll be a lot of time wasted in CI starting/stopping JVMs. Not the end of the world; but might add a minute to the CI time; when it could be more like 10 seconds. #2020-11-06 11:45:42 dominicm Silly question, why not write a short clj file which calls them with the right args? #2020-11-06 11:45:52 dominicm Assuming a functional api #2020-11-06 14:41:13 rickmoynihan > Silly question, why not write a short clj file which calls them with the right args?
Well this is my point. I can definitely do that, and likely will. But I’m only solving it for myself in one project. If you build a jar you almost always want to deploy it.
depstar and deps-deploy would ideally be aligned and integrated around common keys etc.
It looks like deps-library is essentially what that might look like, though with a traditional cli interface not an :exec-fn one. #2020-11-06 14:42:03 rickmoynihan The iteration etc is fairly unique to my project; but the integration of deps-deploy and something like depstar is common enough that it can/should be combined in my mind.
If we had that it would make the job of my iteration/script even smaller (but also I think be widely useful) #2020-11-06 16:34:56 slipset Just to be clear, I’d be happy to move deps-deploy in (almost) any direction that might help solve problems. #2020-11-06 16:36:10 rickmoynihan That’s pretty much my thoughts on what should happen too…
I feel we need some community consensus on what that direction might be though. #2020-11-04 18:25:54 rickmoynihan Anyway that’s me done for the day… but will check back later / tomorrow. #2020-11-04 21:01:10 Eamonn Sullivan Hi all. I'm trying to use a library in my deps.edn that's on our corporate artifactory. This site uses ssl client certificates for authentication. In Scala/sbt (our normal code), we set java options such as -.ssl.keyStore=... -.ssl.keyStorePassword=... in the environment ($SBT_OPTS). But I can't seem to get this to work for clojure. I tried just directly passing these in like -.ssl.keyStore=... but it doesn't seem to work. I get a handshake_failure. Does this sound familiar to anyone? I can build the library I want locally and install it directly in ~/.m2, but that's an awkward way to work. #2020-11-04 21:05:43 ghadi the reason that doesn't work is because -J options apply to the application JVM that gets launched -- your program.
the dependencies themselves are fetched in a previous invocation that doesn't receive those -J options #2020-11-04 21:06:14 ghadi will defer to @alexmiller on how to use custom TLS on the deps fetching #2020-11-04 21:10:26 Eamonn Sullivan Thanks, yeah, that makes sense. I can continue working (installing locally) for now -- just trying to make the workflow a bit less convoluted. #2020-11-06 16:33:10 rickmoynihan @seancorfield: would you take a PR to refactor the bulk of this function into a new function that does everything except call System/exit? #2020-11-06 16:34:15 rickmoynihan oh actually looks like it’s not system-exit that is the problem… but shutdown-agents #2020-11-06 16:35:12 rickmoynihan well the system/exit is also bad as it kills my repl when I make a mistake in the config #2020-11-06 16:36:21 seancorfield It calls System/exit in multiple places. #2020-11-06 16:36:33 rickmoynihan yeah so I’ve seen #2020-11-06 16:36:41 alexmiller I often try to separate the shutdown-agents/System.exit type stuff into a wrapper #2020-11-06 16:36:48 rickmoynihan yeah me too #2020-11-06 16:36:57 rickmoynihan that’s what I’m suggesting #2020-11-06 16:37:37 seancorfield Yeah, I would normally but I haven't quite gotten there yet. #2020-11-06 16:40:59 seancorfield There's also the call to help-and-die in there. I'll give it some thought this weekend. #2020-11-06 16:41:27 rickmoynihan shall I make an issue for it? #2020-11-07 13:32:39 rickmoynihan Thanks sean, sorry I didn’t get around to this yesterday. #2020-11-07 18:44:49 seancorfield NP. #2020-11-07 20:53:40 seancorfield @rickmoynihan release 1.1.133 has this refactoring so you can use hf.depstar.uberjar/run* from the REPL now. Same args as hf.depstar.uberjar/run but it returns a hash map for success/failure. The old entry points all still call (shutdown-agents) or (System/exit 1). #2020-11-07 20:53:51 seancorfield Follow-up in #depstar if you want. #2020-11-06 16:41:44 seancorfield Sure. It'll be a good reminder for me. #2020-11-06 16:42:34 alexmiller another option is to parameterize the "on-exit" function #2020-11-06 16:45:22 borkdude I usually write a -main and main. Main only returns the exit code (or a map with exit code) and -main does System/exit + shutdown-agents #2020-11-06 16:45:30 borkdude I then use main in my tests #2020-11-06 16:46:33 rickmoynihan yeah that’s the same pattern I do… though this isn’t main but an :exec-fn not that it makes much difference. #2020-11-06 16:47:24 rickmoynihan it lets you essentially test command line opts etc #2020-11-06 16:49:04 borkdude yes, this is how I test all my CLIs using the exact same input as from the command line #2020-11-06 16:51:43 rickmoynihan Whilst I’m hear, as you may have seen I’ve been tweaking deps-deploy to support the new -X stuff.
If you’re going to use deps-deploy however; you essentially need to use depstar (or similar), so it’d be good if these tools could be aligned a bit to respect the same keyword vocabulary.
How do people feel about that?
Similarly in the deps-deploy changes I made I thought it would be nice to use :mvn/version as it means the same as in tools.deps. However I also wanted :mvn/artifact-id and :mvn/group-id etc. Would those terms in principle be the same as tools.deps / tools.build etc might use?
Is it legitimate for me to coin keys in the mvn namespace? #2020-11-06 16:52:48 alexmiller to use in what context? #2020-11-06 16:53:12 alexmiller like in calling with -X or something? #2020-11-06 16:53:27 rickmoynihan yeah precisely, my patches to deps-deploy allow you to override the :mvn/version via :exec-args and the cli, but also :mvn/artifact-id etc… #2020-11-06 16:53:46 alexmiller probably won't line up to tools.build but I don't know if you should care #2020-11-06 16:54:35 rickmoynihan well I don’t know what tools.build does… so hard to know if I should care or not 😀 #2020-11-06 17:03:17 rickmoynihan I guess I have a slightly deeper question here though…
Which is essentially how deep does RDF’s influence over best practice around keywords in clojure go?
Rich has mentioned RDF publicly multiple times over the years. And clojure’s namespaced keywords etc are essentially equivalent to predicates etc. The promoted strategy to handling them has been “if they have the same meaning” it’s fine to reuse them (open world, open for extension etc…). However the linked-data world also has a convention of only coining new terms in namespaces you own. However clojure best practice glosses over that here. Does tools.deps own :mvn, is it legitimate for me to coin keywords in that namespace? #2020-11-06 17:03:55 rickmoynihan Also is the concept of keyword vocabularies perhaps a useful one? #2020-11-06 17:03:59 dpsutton What is "RDF"? #2020-11-06 17:08:01 rickmoynihan TLDR; A web-scale datomic-like data model described as a set of W3C standards and practices for how people can build databases to support a global graph of knowledge #2020-11-06 17:11:29 rickmoynihan (Also acknowledged by Rich to be a big influence on the design of clojure and datomic too) #2020-11-06 17:08:09 alexmiller I do not consider tools.deps to "own" :mvn #2020-11-06 17:09:13 alexmiller qualifiers and their usage are contextual. spec argues for sufficient qualification to create a universe of unambiguous attribute definitions #2020-11-06 22:46:40 dominicm Fwiw, I think this would be a valuable article on http://clojure.org. Explaining the purposes of namespacing and when you can skip it, when to use #db/id instead of #com.datomic/id Or #datomic/id. #2020-11-06 17:10:01 alexmiller in more limited contexts (inside your project or inside another appropriately scoped data context), less qualification may be sufficient #2020-11-06 17:10:51 alexmiller deps.edn has a competing design constraint of wanting to be succinct as this is a format people are frequently writing #2020-11-06 17:11:38 alexmiller we ended up leaning more on the latter constraint in that balance #2020-11-06 17:13:49 rickmoynihan Thanks for that @alexmiller, that’s essentially my perception too.
What then do you think of collaborating around common shared definitions like :mvn/version etc and agreeing in principle on other ones to avoid future conflict in areas such as this? #2020-11-06 17:14:17 alexmiller for the purposes of invoking tools, I don't think there's any need to collaborate #2020-11-06 17:14:34 alexmiller and I don't see it as conflict #2020-11-06 17:16:58 rickmoynihan well it’s not a conflict if I chose :mvn/group-id and you choose :mvn/groupid, but it is irritating if we mean the same thing and our terms flow through the same channels.
It would be a potential conflict if we both independently chose :mvn/repository but I defined it as a map, and you a string; and they happened to flow through the same context. #2020-11-06 17:18:17 alexmiller I don't see them flow through the same channels or exist in the same context, which is why I say this is not a conflict #2020-11-06 17:19:05 alexmiller really, I'd say neither of these tools "own" :mvn. use a qualifier you own if you want to own it. #2020-11-06 17:19:31 alexmiller otherwise, don't complain when it conflicts :) #2020-11-06 17:26:54 rickmoynihan Well that’s certainly pragmatic. I’ll take that as permission to follow your example and use :mvn/group-id here. I mean it seems sensible to follow similar conventions for human reasons. #2020-11-06 17:34:22 alexmiller I have no opposition to using that name :) #2020-11-06 17:25:11 seancorfield depstar's arguments to -main are pretty fixed at this point, and I've just named the :exec-args after the long-form command-line arguments I believe. Changing them would break backward compatibility so I don't want to do that. Re: deps-deploy -- I think it's pretty important that pom.xml be the source of truth for the group/artifact/version -- at least insofar as the version that is included in the JAR and uploaded to Clojars with the artifact -- so I don't think it's a good idea to override the version in deps-deploy and end up with the internal POM not matching the external POM. #2020-11-06 17:27:01 vlaaad And then there is me who thinks that pom.xml it's just a temporary file that should be deleted as soon as deployment is done #2020-11-06 17:27:25 seancorfield At this point, I would prefer to defer all of this to tools.build but I don't know whether deployment is in scope for that (we have local mvn-install already in t.d.a but no remote deployment). #2020-11-06 17:27:58 seancorfield I don't want to have to keep maintaining a community tool if there is an official tool from the core Clojure team that does the same job. #2020-11-06 17:28:31 rickmoynihan @seancorfield: yeah that’s fair… I’d much rather overriding the :mvn/version etc happened in depstar. I said this the other day in fact. #2020-11-06 17:28:41 alexmiller I think deployment is one of many things that could be invoked by tools.build #2020-11-06 17:29:08 alexmiller whether that's something provided as part of tools.build or community is a separate question. I've looked at it in depth, have a lot of opinions, have not built it. #2020-11-06 17:29:22 seancorfield @vlaaad Well, the reality is that there's a lot of stuff in pom.xml that other tooling out there relies on, far beyond the minimum that clojure -Spom produces, so you pretty much have to maintain that information somewhere under version control. #2020-11-06 17:32:32 vlaaad Of course, I just think that pom.xml is a horrible storage for this information, since there is a lot of complexity with regards to updating it #2020-11-06 17:33:58 seancorfield Oh totally agree! XML is a horrible source format -- but we're "not supposed to use" deps.edn for this stuff, which is where I would otherwise expect it to be... although with :exec-args etc, that's getting more reasonable and "sanctioned" as appropriate by the core team. #2020-11-06 17:30:45 seancorfield @rickmoynihan Yeah, there are parts of pom.xml that it would be nice for depstar to be able to override but if you override <version>, you pretty much have to override the SCM <tag> to match, because of tooling that uses that. #2020-11-06 17:31:26 rickmoynihan yeah @U09LZR36F mentioned this the other day… that’s trivial to do too though. #2020-11-06 17:32:24 seancorfield A guiding principle for depstar is no additional dependencies so the more functionality it accretes, the more I end up rewriting other tooling into the depstar codebase. #2020-11-06 17:35:25 rickmoynihan Why no additional dependencies? A stable xml lib would be handy, and tools are usually separate from an apps :deps anyway. #2020-11-06 17:35:50 rickmoynihan even tools.deps has dependencies! #2020-11-06 17:36:10 rickmoynihan and clojure 🙂 #2020-11-06 17:37:12 seancorfield Look at the way depstar works and you'll see: it packages up what it finds on the classpath you used to run it, simply excluding itself. #2020-11-06 17:37:56 seancorfield That ensures you get exactly what you expect in the JAR based on the aliases you provide, and mean that you don't have to tell depstar itself how to read/combine aliases/`deps.edn` etc. #2020-11-06 17:38:02 rickmoynihan It also allows you to set an explicit :classpath though right? #2020-11-06 17:38:23 seancorfield Yes, that's an alternative option but I don't want it to be the default. #2020-11-06 17:38:54 seancorfield Lots of people seem to struggle with the other JAR-makers out there and end up at depstar because it is simple and does the obvious, intuitive thing. #2020-11-06 17:40:07 rickmoynihan sounds more like easy to me… simple would be having the :classpath supplied explicitly. #2020-11-06 17:42:27 seancorfield The original version of depstar that I inherited had pretty much no options and just created a JAR from the classpath. It was about as simple as it could be. Adding AOT and main-class and other stuff has made it less simple as a tool, but still easy 🙂 #2020-11-06 17:50:49 rickmoynihan yeah I understand, and there’s nothing wrong with easy.
It just sounds like that’s a requirement for the uberjar side of depstar; but the library side is I think a far simpler problem. Perhaps it should be split into two separate libraries, depstar.jar and depstar.uber? #2020-11-06 17:52:43 seancorfield It could be refactored to keep those two things separated. Lots of things could be done with depstar but, right now, I'm waiting to see where tools.build ends up. #2020-11-06 17:53:17 seancorfield Like I say, I don't want to keep maintaining a community tool when there's an official core Clojure team tool that does the "same" job. #2020-11-06 17:53:48 seancorfield If it seems like tools.build won't handle the cases that depstar can handle, then I'll be happy to invest a lot more time into it. #2020-11-06 17:54:53 rickmoynihan that’s fair enough 🙂
but I need solutions now, not in 2+ years 🙂
Being able to build and deploy a library into a private s3 bucket is a common requirement that’s far more awkward than it should be right now. #2020-11-06 17:30:49 alexmiller the code under -Spom has been refactored to make it more amenable to having more options/features and I think longer term that's going to be something we push to primarily being done as part of a build, with probably ever-increasing scope to refine as part of a build #2020-11-06 17:30:54 rickmoynihan Yeah… For my needs I think treating pom.xml as a template, and moving my override feature from deps-deploy into depstar would make most sense. #2020-11-06 17:31:50 alexmiller there are all sorts of special cases you might want to handle in making a pom for deployment #2020-11-06 17:32:29 alexmiller many require some kind of data manipulation you'll want to do during a build #2020-11-06 17:33:54 alexmiller just fyi on tools.build - it exists, it has a lot of functionality but we are trying to finalize a lot of the details around how tasks are defined, stitched together, invoked, etc. b/c that kind of affects everything, we're delaying the public release of it but we'll get there. #2020-11-06 17:35:08 seancorfield And I'm still going to bleat on and on about some sort of built-in solution for either the monorepo case or where folks want to combine/insert an extra "master" deps.edn file into the t.d.a mix that everything should then use 🙂 #2020-11-06 17:35:33 seancorfield (ideally, in t.d.a so that tooling could rely on it easily) #2020-11-06 17:36:35 seancorfield Some sort of "include" or "reference" functionality in deps.edn would work for that -- but I think you've said the core team are opposed to going down that path @alexmiller? #2020-11-06 17:37:14 alexmiller my initial lean would be towards an additional deps.edn source but haven't talked about it enough #2020-11-06 17:40:17 seancorfield That would satisfy me too. I'd like to get away from my current CLJ_CONFIG hack, and I'd like individual developers to be able to keep using their own ~/.clojure/deps.edn file for dev/test stuff without us needing to repro a common core set into our "master" deps.edn file 🙂 #2020-11-06 17:42:40 rickmoynihan FWIW I think this would be useful too. #2020-11-06 18:01:22 rickmoynihan @alexmiller: Which problem does tools.build solve more of? Orchestrating arbitrary tasks together, or providing ways to build jars, deploy stuff etc?
My feeling is that it’s more like make than a way to build jars? Is that right? Or is it both?
I appreciate you don’t want to go into too much detail; but it’d be useful to know if depstar etc might be replaced wholesale by it; or whether something like depstar would just integrate with it? #2020-11-06 18:58:57 alexmiller it is both a framework and a set of built-in tasks, so both, and could probably replace a number of existing tools and/or integrate them #2020-11-06 18:03:01 rickmoynihan Ok sorry folks this is going to have continue another time…
Many thanks to everyone involved in the discussion. I hope you all have fab weekends. 👋 👋 #2020-11-06 19:01:51 alexmiller I would love to see the built-ins become de facto defaults and make them great with community help #2020-11-06 19:41:45 dominicm I'm very curious to see how/if caching is handled. Notoriously difficult task. #2020-11-06 19:45:27 alexmiller caching of? #2020-11-06 20:10:43 vlaaad I would guess caching of task results #2020-11-06 20:11:53 dominicm E.g. make #2020-11-06 20:12:01 alexmiller if you mean like compiled classes, that's one thing we're still thinking through #2020-11-07 14:00:02 alexmiller Don’t get too excited, very experimental #2020-11-07 14:04:58 dominicm I just loved the approach :) #2020-11-07 14:05:41 rickmoynihan Currently digging through depstar etc in a spare few minutes, and I’m sat wishing the community had just adopted Stu Halloway’s lancet for builds 12+ years ago 😆
Seriously though, I know ant was a bit of a pain at times but ant as sexps/edn with some deployment stuff would be miles better than what we have right now. #2020-11-07 14:06:39 alexmiller I still like many things about ant #2020-11-07 14:06:45 rickmoynihan yeah me too #2020-11-07 14:07:58 alexmiller The only thing maven did right was standardize deps and repos. But that was so big, it won. #2020-11-07 14:09:22 rickmoynihan Yeah was literally about to say just that, and…
The other problem with ant was that every build was different; but it was expressive enough and pretty efficient and the core tasks worked well.
Maven was easier for simple library builds; but far harder for anything that needed to do more than compile, dist, deploy and document. #2020-11-07 14:10:57 rickmoynihan You can integrate resolution and deployment with ant right? Isn’t that what ivy was? I never set that up all those years back. #2020-11-07 14:11:52 dominicm Gradle seemed very sophisticated when I looked at it. #2020-11-07 14:13:02 rickmoynihan I never used it; looked like Rake… essentially ant built as a dsl in groovy right? #2020-11-07 14:27:03 borkdude Today a new version of clj-kondo was released that includes deps.edn linting. You may have to update flycheck-clj-kondo if you are using emacs integration. #2020-11-07 14:52:06 alexmiller I have known many people preaching gradle over the years and have given it an honest shot a couple times. I found it inscrutable. It’s probably just me but I just could not understand how to do anything. I’m the same way with ruby though - I just find it baffling. Just give me some parens man. #2020-11-07 15:02:25 borkdude What I found most baffling about ruby was rails (I know that's not ruby, but very common): I didn't now where to look or follow things, things just magically came from random places. #2020-11-07 15:03:39 borkdude At least ruby doesn't have significant whitespace. Don't get me started on that! </rant> #2020-11-07 15:04:52 dominicm My recollection of Gradle was seeing the sheer investment someone had made to integrate ClojureScript with the build system so the caching/dependency system would work.
May not have been Gradle, might have been something else. #2020-11-07 17:12:04 flowthing Gradle is fine, though quite complex. One big downside it has compared to Ant, IMO, is that it doesn’t have immutable properties. #2020-11-07 17:39:53 dharrigan We use gradle a lot at work, more specifically gradle with the kotlin dsl. One can declare vals which are immutable (or vars, which are mutable) #2020-11-07 17:40:17 dharrigan val foo = "hello" would make foo immutable. #2020-11-07 18:41:59 flowthing Right, the Kotlin DSL didn’t exist back when I most used Gradle. Still, I don’t think that helps if you want to define a property in the CLI invocation such that it’s not overwritten somewhere down the chain. #2020-11-07 17:40:53 dharrigan I think the kotlin dsl is winning (esp for android and spring based projects). Others may still use the groovy dsl. #2020-11-09 09:49:08 Eamonn Sullivan Just a reminder that I had a practical question ☝️ about how to use an artifactory with ssl client certs authorisation. Our internal artifactory requires -.keyStore and -`http://Djavax.net.keyStorePassword` to be specified. How do I do that for the deps fetching? Is that possible, at the moment? #2020-11-09 12:29:30 practicalli Sorry, after finding your original question, my previous answer doesnt make sense... #2020-11-09 12:56:59 alexmiller This is not possible right now with clj, but feel free to ask at https://ask.clojure.org and I will turn that into a ticket #2020-11-09 15:31:42 alexmiller thx #2020-11-09 15:31:58 alexmiller there are some other cases where passing java opts to dep resolution would be useful #2020-11-09 09:58:42 borkdude @eamonn.sullivan Maybe not exactly what you are looking for, but if tools.deps doesn't have an answer right now: I've got a port of the clojure CLI script in Clojure itself. It supports setting a bunch of java properties for dep fetching through environment variables. Right now only proxy settings, but your variables may also be appropriate to support.
Take a look at https://github.com/borkdude/deps.clj. #2020-11-09 10:00:18 borkdude You can run with the JVM, with babashka or as a graal binary. You can even run it as a clojure dep (since it's just normal clojure code) which is kind of meta. #2020-11-09 11:15:10 Eamonn Sullivan Thanks, @borkdude, I will take a look. #2020-11-11 03:15:51 nate Where are all the places you can use an alias keyword to interpolate data from aliases? I've seen examples in :paths and :exec-args, and I'm wondering how far the rabbit hole goes... #2020-11-11 03:18:00 alexmiller those are it right now #2020-11-11 03:21:00 nate cool #2020-11-11 03:21:15 nate thank you #2020-11-11 09:30:30 henryw374 I'm looking for a tool that will help pull out unnecessary :exclusions - ie keeping the set of resolved dependencies unchanged for some combination of aliases. anyone know of such a thing? thanks #2020-11-11 12:09:53 henryw374 another thing I'm looking for is a documented description of how deps chooses which version of a dependency to use. I expect it's there but I just can't see it on http://clojure.org and so far google has failed #2020-11-11 12:27:08 henryw374 thanks #2020-11-11 13:08:16 alexmiller I did a talk at the conj about it, that’s probably the best description right now. The summary though is that it tries to use the newest version it can. #2020-11-11 14:46:48 andy.fingerhut "the newest version it can" could be easily misinterpreted as "the newest version available on Maven central and/or Clojars", e.g. similar to behavior of using a -SNAPSHOT or -RELEASE version. #2020-11-11 14:47:52 andy.fingerhut Would it be accurate (but slightly longer) to say "the newest version among all of those versions explicitly requested by at least one of your dependencies (but top level can override that)" ? #2020-11-11 14:53:55 alexmiller yes #2020-11-11 13:09:02 alexmiller But top level dep versions always win #2020-11-11 13:17:34 henryw374 gottit! thanks #2020-11-11 14:57:33 imre is there any way to tell the clj command to ignore my custom .clojure/deps.edn? #2020-11-11 14:58:49 imre ah, I may have found it, -Srepro #2020-11-14 16:16:55 alexmiller no real reason - the internals have been refactored to make this feasible but really I expect this to probably go away in favor of doing this via tools.build (which uses most of this same code) #2020-11-14 16:18:26 rickmoynihan ok in that case my tool wasn’t a complete waste of time 😂 #2020-11-14 16:33:40 vlaaad Ah, I hoped tools-build would have a solution for xml madness, that's great to hear! #2020-11-14 16:39:51 seancorfield @rickmoynihan...and this is why I've been pushing back on the depstar discussion until I see what tools.build actually brings to the table: I don't want to do work that would be made redundant by tools.build because I'd much rather use an "official" tool chain, maintained by Nubank, than have to keep maintaining and working with ad hoc third party stuff 🙂 #2020-11-16 15:33:50 rickmoynihan yup I’ve never questioned that… I’m just trying to improve this area for myself and maybe others in the interim #2020-11-15 13:38:33 borkdude borkdude/babashka {:git/url ""
:sha "41ede17be04965c975e1c0f00e221b02bc5cf879"}
I'm getting:
$ clojure -Sforce -P -A:native
Error building classpath. Manifest type not detected when finding deps for borkdude/babashka in coordinate {:git/url "", :sha "41ede17be04965c975e1c0f00e221b02bc5cf879"}
Why is this? With :deps/manifest :deps added it "works" but it can't find resources in that project #2020-11-15 13:40:58 borkdude Maybe it has issues with projects with git modules? #2020-11-15 13:41:11 borkdude I'll just use the mvn/version one #2020-11-15 14:28:35 alexmiller There is a known issue with git submodules #2020-11-15 14:29:10 alexmiller There’s a ticket for that, not sure if that’s what you’re seeing #2020-11-15 15:16:48 borkdude No worries, I already solved it a different way #2020-11-16 18:52:57 borkdude @jplaza Hmmm. You could spit out the classpath to a file and then start Clojure using java -cp @classpath-file clojure.main but this is a bit spartan maybe. #2020-11-16 18:53:30 alexmiller there is a ticket with some work on it but I don't know that they are workarounds currently #2020-11-16 18:56:18 borkdude @jplaza
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "52303d203936273637121f10026062636b"}, :content ("[email protected]")} #2020-11-16 18:57:15 borkdude What this does: java reads the classpath from a file here. #2020-11-16 18:57:30 borkdude Tools.deps / clojure CLI might leverage the same workaround #2020-11-16 18:57:46 alexmiller seems like it could easily do so since it always has a classpath file already :) #2020-11-16 18:58:25 borkdude yeah #2020-11-16 18:59:23 alexmiller this would actually be trivial #2020-11-16 18:59:39 alexmiller if you wanted to hack your ClojureTools.psm1 and try it #2020-11-16 19:01:56 alexmiller basically where you have -classpath $CP if you changed that to "@$CP" (probably not the right syntax but that's the idea) in like line 396 #2020-11-16 19:02:22 alexmiller $CP here is the name of a classpath file #2020-11-16 19:03:35 alexmiller the exec case is maybe the hardest as it adds the special exec jar: -classpath "$CP;$InstallDir/exec.jar" so might need to emit a well-known variant for that case or something #2020-11-16 19:04:03 alexmiller if someone wanted to work on a patch, would be happy to integrate that #2020-11-16 19:04:49 borkdude you would have to append $InstallDir/exec.jar to the classpath file and then read that. Probably better to append it to some other file classpath-exec and then use that one #2020-11-16 19:05:25 borkdude afaik you can only use one @ and not in combination with non-file classpath #2020-11-16 19:06:32 alexmiller yes that would be my expectation. I'm suggesting that right now we write 12341234.cp and we could also write 12341234-exec.cp (which included the exec jar) and use that instead for the -X case #2020-11-16 19:06:40 borkdude yes #2020-11-16 19:10:14 borkdude You might want to check compatibility of this over several java versions #2020-11-16 19:14:09 borkdude The java docstring doesn't mention this explicitly #2020-11-16 19:16:29 alexmiller been around for a long time #2020-11-16 19:16:51 alexmiller if it works on Java 8, then it's ok by me :) #2020-11-16 19:17:53 borkdude > been around for a long time
where do you find docs about this? #2020-11-16 19:18:40 alexmiller looks like maybe java 9? #2020-11-16 19:18:52 borkdude I'll try java 8 #2020-11-16 19:19:21 borkdude nope, didn't work #2020-11-16 19:20:21 alexmiller I think maybe it was added in java 9 based on my googling. I thought it was older than that #2020-11-16 19:21:23 alexmiller well, could make it version dependent - if java version is 9+, use @ #2020-11-16 19:21:52 borkdude Right, it seems the @argument file thing applies to every option #2020-11-16 19:21:52 alexmiller or maybe invert, if Java version is 8, don't use @ #2020-11-16 19:23:29 borkdude Yeah, that makes sense. That will assume you will not re-use your cache for different java versions, so maybe include the java version in the cache hash if you haven't already #2020-11-16 19:24:32 alexmiller I don't think you need that #2020-11-16 19:25:02 alexmiller the files are the same either way, it's just whether or not you can use it via @ #2020-11-16 19:25:37 borkdude right. how will the CLI know which java version is used? #2020-11-16 19:25:49 alexmiller by calling java -version ? #2020-11-16 19:26:08 borkdude $ time java -version
java -version 0.07s user 0.06s system 94% cpu 0.136 total
#2020-11-16 19:27:06 alexmiller what's the error if you use the @file on java 8? #2020-11-16 19:27:22 borkdude $ java -cp @classpath clojure.main
Error: Could not find or load main class clojure.main
#2020-11-16 19:27:44 alexmiller not too distinguishable then #2020-11-16 19:28:03 borkdude Maybe this can be controlled via an environment variable #2020-11-16 19:28:05 alexmiller I was wondering if you could just assume it would work, detect the failure and retry #2020-11-16 19:28:11 borkdude CLOJURE_CP_FILE=true #2020-11-16 19:28:21 alexmiller yeah, it could be an env var too. I was trying to avoid that :) #2020-11-16 19:28:38 alexmiller might be the cleanest though #2020-11-16 19:29:00 borkdude for those experiencing this problem, they would set it, others won't #2020-11-16 19:29:11 alexmiller that's fine #2020-11-16 19:29:30 borkdude and only on Java 9+. #2020-11-16 19:30:07 borkdude @jplaza what Java version are you using? #2020-11-16 19:32:12 borkdude @alexmiller alternative dirty workaround: use add-lib to load the libs later on ;) #2020-11-16 19:34:31 alexmiller bleh #2020-11-16 19:34:59 borkdude yeah, I said dirty, but it's always good to list alternatives, even when they're bad ;) #2020-11-16 19:36:11 borkdude another alternative: won't fix until Java 8 isn't supported anymore :) #2020-11-16 19:36:24 alexmiller who knows if that day will ever come #2020-11-16 19:37:41 borkdude another alternative: reverse engineer the java command line tool and build a nicer front-end which supports this for older java versions ;) #2020-11-16 19:41:17 jplaza Thanks for the fast reply @alexmiller @borkdude! I’m helping a teammate with that issue. She has java 9. #2020-11-16 19:42:25 jplaza I told her the problem is that she’s using Windows haha! She’s installing Windows Subsytem for Linux #2020-11-16 19:43:18 borkdude #2020-11-16 19:46:50 jplaza 😆 #2020-11-16 19:47:08 borkdude > Windows command-line length limit
Does this apply to cmd.exe, powershell.exe or is it a conhost limit? Do you experience the same problem with the new Windows Terminal? #2020-11-16 19:48:34 jplaza Not sure but happens with both cmd and powershell #2020-11-16 19:48:55 borkdude wsl2 is pretty good ;) #2020-11-16 19:51:55 seancorfield Yeah, I gave up on the whole Powershell thing and just do everything on WSL2 now. Esp. with VS Code's Remote-WSL2 extension. #2020-11-16 19:53:08 borkdude Have you also tried portal btw? #2020-11-16 20:01:04 seancorfield Yes, but I prefer to dock a GUI next to my editor rather than switch back and forth between an editor and a browser. #2020-11-16 19:52:20 seancorfield (I run VcXsrv aka Xlaunch so I can start X11-based GUIs from WSL2, such as REBL or Reveal) #2020-11-16 20:51:12 jplaza Cool! Any tips on using Calva + WSL2? #2020-11-16 20:53:01 seancorfield @jplaza I use Clover with VS Code. I like working with a plain Socket REPL in processes -- I avoid nREPL and all that stuff. Clover is "Chlorine for VS Code" and I was previously using Chlorine on Atom. #2020-11-16 21:00:51 borkdude @jplaza Many people use Calva. Check out #calva. There's a lot going on there. #2020-11-16 21:01:28 borkdude @jplaza Also check out out the clj-kondo plugin for VScode </shameless-plug> #2020-11-16 22:34:34 jplaza We use clj-kondo in our team! Awesome tool! I actually mentioned our startup in the issue so you can list us there 😉 #2020-11-16 21:32:40 gmorpheme I assumed clojure -X:deps mvn-pom :lib 'foo/bar' :version '"0.0.1-SNAPSHOT"' would generate generate me a POM with the specified details but I’m not having any luck - I just get defaults. (Maybe mvn-pom doesn’t pass the args on through to sync-pom?). Was this not an intended usage? Or am I missing something? Or just an oversight? #2020-11-16 21:57:28 alexmiller it does not pass those through right now #2020-11-16 21:57:53 alexmiller mvn-pom may go away in favor of tools.build once that is out so I'm resisting the urge to make it better #2020-11-17 09:11:29 gmorpheme understood, thanks #2020-11-17 09:29:03 rickmoynihan hot off the press so YMMV, I’ve not even properly announced it yet. #2020-11-17 09:41:42 gmorpheme Thanks - works for me if I generate the POM with tools.deps then update the version with pom-update. I needed to add :pom/write-strategy :overwrite :pom/input-file '"pom.xml"' explicitly though. #2020-11-17 11:02:55 rickmoynihan oh yes thanks that’s intentional, I just forgot to add that to the README.md when I wrote it up last night.
I’ll do that now.
Thanks for letting me know. #2020-11-17 11:03:02 rickmoynihan Fixed. #2020-11-17 11:03:30 rickmoynihan My recommendation is that you include those in your deps.edn rather than via command line, as they won’t vary across calls. #2020-11-17 11:03:57 rickmoynihan See the updated README.md #2020-11-16 22:12:36 pez @jplaza if you find out stuff about using WSL2 with Calva, please consider updating this page: https://calva.io/wsl/ #2020-11-16 22:35:09 jplaza Will do #2020-11-17 14:53:35 imre Hey folks. I just ran into a problem at deps resolution: Error building classpath. Could not find artifact gcardone:junidecode:jar:0.2 in central ()
My project is not directly referencing this lib and I don't know where in the dep tree it is referenced from. Is there any way to trace this with the clojure command line? #2020-11-17 14:55:16 borkdude clojure -Stree #2020-11-17 14:55:53 imre Unfortunately that fails with the same error without printing anything useful. So do -Spath and -Strace #2020-11-17 14:56:25 borkdude oh right :) #2020-11-17 14:56:32 borkdude then bisect? #2020-11-17 14:57:04 borkdude maybe also try -Sforce which will force recreation of classpath #2020-11-17 14:57:45 imre bisect it is then #2020-11-17 14:57:49 imre 😞 #2020-11-17 14:57:57 imre force does not help #2020-11-17 14:59:36 dominicm I'm surprised trace doesn't output anything. That would be pretty useful for this case. #2020-11-17 14:59:55 dominicm Is the secret verbose flag still a thing the? #2020-11-17 15:00:09 alexmiller no #2020-11-17 15:00:34 alexmiller there isn't a good answer for this right now other than tracing down the deps tree semi-manually #2020-11-17 15:00:50 alexmiller I've been looking at some new tools for questions like this #2020-11-17 15:01:17 alexmiller or you could -Spom and mvn dependency:tree #2020-11-17 15:01:25 alexmiller might get more info that way #2020-11-17 15:01:30 imre hmm could try that yeah #2020-11-17 15:01:53 imre well, that also fails with the same I'm afraid #2020-11-17 15:02:52 alexmiller maybe mvn -X dependency:tree ? #2020-11-17 15:03:31 imre Sorry, I meant -Spom fails #2020-11-17 15:04:35 alexmiller oh, right :) #2020-11-17 15:21:52 imre found it in the end by reverting and updating deps one by one